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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 


WASHINGTON.  D.C.  20301 


RESEARCH  AND 
ENGINEERING 


DoD/JOINT  SERVICES 

PRODUCTION  READINESS  REVIEW  CONFERENCE 


Introduction 


This  document  is  the  official  proceedings  of  the  DoD/ Joint  Services  Production  Readiness 
Review  (PRR)  Conference  held  at  Wright  Patterson  AFB,  Ohio,  on  19  and  20  November 
1980.  The  conference  convened  DoD  personnel  from  the  three  Military  Departments  and 
the  Defense  Logistics  Agency  who  have  or  will  shortly  participate  in  PRR's.  The  purposes 
of  the  conference  were  to  exchange  "lessons  learned"  and  to  explore  ways  to  make  PRR's 
more  cost  beneficial  and  more  contributory  to  the  DoD  materiel  acquisition  process. 
Subsequent  production  management  conferences  are  planned  to  address  other  key 
production  functions. 

Among  the  areas  identified  for  greater  emphasis  in  future  PRR  activity  were:  (1) 
production  cost  reduction;  (2)  determining  software  readiness;  (3)  use  of  Government  plant 
activity  personnel;  and  (4)  development  of  quantitative  measurements  of  production 
readiness. 

These  proceedings  include  speaker  presentation  material  and  narratives,  a  list  of 
attendees,  and  the  major  observations  and  continuing  actions  synthesized  from  the 
discussion  periods  held  throughout  the  conference. 

It  is  strongly  urged  that  all  attendees  give  attention  to  the  request  for  data  needed  to 
continue  the  work  of  the  Defense  Product  Engineering  Services  Office  on  PRR 
quantitative  measurements  and  respond  appropriately.  This  request  is  contained  on  page 
295  in  the  proceedings. 

The  assistance  of  the  Air  Force  organizational  elements  and  personnel  who  hosted  and 
administered  this  conference  is  gratefully  acknowledged. 


Office  of  Deputy  Under  Secretary  of 
Defense  (Acquisition  Policy) 
Directorate  of  Major  Systems  Acquisition 
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KEYNOTE  SPEECH 


Truxtun  R.  Baldwin 

Staff  Asst,  for  Production  Management 
Office  of  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Policy) 
Directorate  of  Major  Systems  Acquisition 


Good  morning.  This  is  the  DoD  Joint  Services  Production  Readiness  Review  Conference. 
Our  purpose  §here  today  is  to  share  our  experiences  or  "lessons  learned  in  conducting 
PRRs  with  the  objectives  of  first  improving  the  PRR  process  itself,  making  it  more  cost 
beneficial,  and  secondly,  making  the  PRR  contribute  more  to  our  conduct  of  the 
production  phase  and  a  better  acquisition  process  overall. 

I  would  like  to  get  three  points  across  in  starting  the  conference.  One,  I  would  like  to 
recognize  the  contribution  that  all  of  us  who  are  engaged  in  the  production  function  are 
making  here  to  the  Defense  effort.  I  am  not  talking  just  about  the  impact  of  the  PRR  on 
the  decision  process  itself,  but  the  entire  production  function;  the  impact  it  has  on  the 
acquisition  process  as  a  whole.  We  are  well  aware  that  producing  a  system  for  inventory 
is  the  ultimate  goal  of  the  acquisition  process.  As  far  as  money  is  concerned,  the 
production  phase  amounts  to  approximately  half  of  the  Defense  budget  and  is  about  three 
times  what  is  spent  in  R&D.  Time  wise,  the  successful  program  is  in  production  much 
longer  than  it  is  in  development.  Considering  time  and  dollars  and  objectives  the 
production  phase  is  where  the  rubber  meets  the  road. 


The  current  administration  had  as  one  of  its  objectives  elevating  the  production  phase  at 
the  OSD  staff  level  when  they  put  the  production  function  under  the  R&D  program 
directors.  This  made  a  kind  of  continuum  in  responsibility  of  developing  the  system  and 
transitioning  it  into  production.  I  see  that  the  Services  and  their  staff  reorganizations 
have  done  much  the  same  thing.  What  the  new  administration  will  do  we  can  only  guess; 
but  I  believe  the  emphasis  will  remain  on  production  certainly  and  in  producing  those 
systems  we  have  been  developing  during  the  70's  and  getting  them  fielded  in  the  80s. 

Note  we  have  revised  our  two  top-level  system  acquisition  documents;  DoDD  5000.1  and 
DoDI  5000.2.  A  casual  look  through  them  will  find  a  very  heavy  emphasis  on  how  to 
manage  the  production  phase. 

We  in  the  production  community  have  our  own  Defense  Directive  5000.34  whicl|  s*r*s  *ort|] 
the  policy,  assigns  responsibilities  and  specifies  what  will  be  presented  to  the  DSARC  and 
considered  at  each  program  milestone;  not  just  production  ones,  but  each  phase  along  the 
way.  As  specified  the  three  Services  have  developed  focal  points  to  help  administer  the 
production  phase  of  programs. 


Our  subject  today  the  PRR,  is  the  culmination  of  our  involvement  in  this  acquisition 
process  and  caps  our  participation  throughout  the  development  phase.  We  have  a  Defense 
Instruction  5000.38  on  production  readiness  reviews.  It  provides  a  checklist  and  kind  of  a 
uniform  concept  or  approach  to  conducting  the  PRR.  As  you  know  PRRs  are  required  in 
support  of  all  production  milestone  decisions;  both  limited  and  full  production.  As  I  was 
talking  to  one  gentleman  here  just  awhile  ago,  it  really  provides  a  rallying  point  and  is  an 
authoritative  measure  to  get  us  involved  early  in  the  program,  early  in  the  development 

stage. 
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Just  a  word  about  the  mechanics  of  the  OSD  level  involvement  in  PRRs.  Drawing  upon 
the  Service  conducted  PRRs  our  DPESO's  organization  writes  an  independent  production 
readiness  assessment  (PRA)  which  is  submitted  to  the  OSD  R&E  Program  Director  prior  to 
the  DSARC.  This  same  letter  report  is  given  to  the  Defense  Acquisition  Executive,  Dr. 
Perry,  who  is  the  DSARC  chairman.  The  letter  report  goes  in  the  blue  book  which  is 
presented  to  each  of  the  DSARC  principals.  This  PRA  report,  as  we  call  it,  weighs  each 
critical  issue  which  impacts  the  decision  from  a  production  standpoint.  It  is  analogous  to 
the  CAIG  cost  and  the  T&E  test  and  evaluation  reports  which  are  also  special  DSARC 
reports.  So  you  see  the  production  status  is  presented  for  consideration  on  a  more  or  less 
equal  footing  with  the  test  and  the  costing  parts  of  the  decision  making  process.  We've 
had  in  Mr.  Church,  who  has  recently  left  as  our  Deputy  Under  Secretary,  a  strong  voice  in 
bringing  our  concerns  and  issues  to  the  DSARC  process.  He  hasn't  been  hesitant  in  voicing 
our  position.  The  highlights  of  production  readiness  assessments  are  reflected  in  the 
Secretary  of  Defense  Decision  Memoranda,  which  come  out  following  the  DSARC. 

Production  matters  are  becoming  increasingly  involved  in  other  key  acquisition  decisions 
and  determinations.  For  example,  OSD  conducted  a  recent  F-18  program  review  for  the 
SEC  DEF,  wherein  our  office  and  DPESO  played  an  important  part  on  two  of  the  panels. 
In  addition  to  updating  the  production  readiness  area,  we  provided  technical  opinion  on 
producibility  problems  which  the  aircraft  was  having  with  its  landing  gear  and  displays  and 
on  a  composite  bonding  problem.  We  also  participated  in  the  cost  panel  on  that  review. 
This  latter  input  is  quite  important  because  one  of  the  themes  I'd  like  to  get  across  to  you 
is  that  we  want  to  be  more  involved  in  cost  aspects.  Our  input  to  that  panel  was  to 
identify  critical  areas  and  suppliers  where  the  cost  of  the  program  was  of  high  risk. 

At  the  OSD  level,  we  examine  periodically  various  industrial  base  problems  and  get 
involved  in  the  acquisition  strategy  of  programs  particularly  in  establishing  or  maintaining 
competition  during  the  production  phase.  DPESO  has  made  a  number  of  surveys  on 
alternate  sourcing  various  major  components  of  systems. 

The  second  point  I'd  like  to  get  across  is  this  urgency  of  paying  more  attention  to  the  cost 
factor  when  we  run  our  PRRs.  You  are  all  aware  of  the  very  large  procurement  bow 
waves  which  we  have  where  the  annual  funding  requirements  and  backlog  of  previous 
years'  requirements  which  we  did  not  fund,  far  exceed  the  amount  of  money  which  we  can 
expect  to  get  in  each  annual  procurement  appropriation.  One  Defense  Science  Board 
report  said  that  the  requirements  each  year  were  a  third  larger  than  the  amount  of  money. 
Another  DSB  report  said  that  we'd  have  to  reduce  the  unit  cost  of  equipment  forty  percent 
to  be  able  to  buy  everything  we  need.  If  we  look  at  the  prospects  of  getting  more  money 
each  year,  historically  we  find  that  the  trend  in  defense  spending  has  been  remarkably 
level.  In  conducting  PRRs  and  everything  else  we  do  we  should  try  to  help  bring  this  fiscal 
affordability  problem  under  control.  Affordability  has  been  often  described  as  our  most 
pressing  defense  problem,  manifesting  itself  in  deferrals  and  stretch-outs  and  the  resulting 
cycle  of  lower  quantities  and  higher  unit  costs.  During  a  PRR  we  should  determine  where 
the  design  is  not  stable,  where  we  can  expect  production  changes  and  retrofit  costs,  look 
for  areas  where  MM&T  is  lacking  which  would  give  us  lower  yields.  Look  for  facilities 
that  are  behind  schedule  and  where  we're  going  to  have  costly  work  around.  One  area  that 
PESO  is  concentrating  on  now,  is  identifying  these  higher  cost  critical  suppliers, 
contractors  who  have  had  overruns  during  development  or  prototype  production 
difficulties  and  may  try  to  recover  their  costs  in  production. 

The  PRR  criteria,  called  guidelines  in  the  back  of  DoDI  5000.38,  do  require  an 
examination  of  the  contractors'  cost  projections,  and  the  justifications  for  it.  In 
conducting  the  PRR  we  need  to  assure  ourselves  that  these  cost  projections  have  been 
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made  and  be  sure  the  data  is  available.  When  we  do  this,  we  can  expect  some  static  from 
the  cost  communities.  They  may  consider  this  an  encroachment  on  their  cost  estimating 
tasks.  The  counter  to  that  is  we're  not  trying  to  make  estimates  of  cost  but  are  looking 
for  areas  where  the  projections  of  cost  estimates  may  be  weak  because  of  problems  in 
capacity  and  producibility.  We  served  this  function  in  the  F-18  program  review  that  I  just 
mentioned.  The  cost  people  at  OSD,  in  fact,  solicited  our  help  in  this  cost  aspect. 

Let  me  quote  from  DoDD  5000.2,  on  the  subject  of  production  planning.  "From  the  early 
phases  of  the  program  consideration  should  be  given  to  the  costs  of  production,  including 
total  government  investment  required  to  insure  adequate  production  facilities,  availability 
of  critical  materials  and  capability.  Affordability  (there's  that  13-letter  word  again)  must 
be  considered  in  production  planning.  The  program  manager  shall  also  consider  means  to 
increase  the  possibilities  for  competition  during  production."  If  we  look  at  DoDD  5000.34, 
we  see  that  the  contractors'  plans  for  cost  reduction  during  production  are  to  be  reviewed 
at  DSARC,  Milestone  III.  I  believe  we  need  to  emphasize  this  area  more.  We  need  to 
insure  ourselves  that  the  contractors  do  have  adequate  and  active  cost  reduction 
programs. 

We  mentioned  competition.  We've  got  to  pave  the  way  for  initiating  or  maintaining 
competition  during  production.  This  is  a  matter  of  public  law,  in  addition  to  DoD  policy. 
We  need  to  eliminate  or  flag  the  adverse  cost  impacts  of  program  disruptions,  such  as 
stretch-outs.  These  really  kill  us  on  cost  as  you  well  know.  Jack  Bemis  is  one  of  our 
morning  speakers.  He's  done  some  pioneer  work  in  experience  curves  in  trying  to  quantify 
the  change  in  unit  production  cost  as  a  function  of  rate. 

Lastly,  let  me  give  you  a  few  questions  or  issues  which  I  believe  are  germane  to  our 
conference.  As  we  proceed,  lets  look  for  ways  that  we  can  get  ourselves  involved  earlier 
in  the  acquisition  cycle.  This  morning  we  will  be  looking  at  the  cost  of  running  PRRs  and 
we  should  ask  ourselves  if  we're  spending  the  right  amount  of  money.  Is  everything  that 
we're  doing  really  cost  effective,  including  the  items  of  contractor  cost?  How  do  we 
make  PRR  preparations  smoother  for  the  contractor  so  that  we're  not  a  burden  and  we 
work  jointly  on  identifying  and  solving  these  production  problems?  We'll  be  talking  a  little 
bit  later,  about  improving  the  credibility  of  our  PRR  findings  by  better  quantifying  the 
results.  And  finally,  if  we  find  ways  to  improve  the  PRR  process  how  should  we  go  about 
changing  our  policy  and  disseminating  new  guidance  to  the  DoD  components. 

Let  me  now  provide  a  little  guidance  and  instruction  on  how  we're  organized  to  run  the 
conference.  We  have  a  very  crowded  schedule.  We  wanted  to  give  time  to  each  individual 
who  wanted  to  address  the  conference,  and  we  managed  to  accommodate  everyone.  At 
the  same  time  we  wanted  to  have  ample  periods  for  discussion.  We  have  a  little  over 
three  hours  for  open  discussion  for  everybody  to  sound  off.  Also,  we  left  time  to 
summarize  the  results  and  identify  any  continuing  actions  we  want  to  carry  on  after  the 
meeting. 

To  help  me  in  the  administration,  we  have  appointed  a  panel  member  from  each  of  the 
three  Services  and  from  DPESO.  It  will  be  the  panel's  function  to  monitor  the  proceedings 
and  identify  key  points.  I've  asked  them  to  organize  the  presentation  of  the  speakers  from 
their  particular  activity  and  to  introduce  these  speakers.  I've  also  asked  them  to  help  me 
with  the  summary  comments  and  the  wrap-up. 

Let  me  introduce  them  now.  First  I'd  like  to  introduce  Bob  Bidwell,  who  is  the  Director  of 
DPESO.  Representing  DPESO  on  the  panel  is  Bill  Jeffers,  who  is  the  Chief  of  the 
Production  Engineering  Division.  Fred  Michel  is  the  Acting  Chief  of  the  Office  of 
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Manufacturing  Technology,  DARCOM.  Bill  Oaks,  the  Manager  of  the  Navy  PESO, 
NAVMAT.  Cal  Ditrick,  is  Chief  of  the  Manufacturing  Engineering  Division  of  Air  Force 
Systems  Command. 

As  far  as  instructions  to  the  presenters  are  concerned,  I  ask  you  to  really  hit  the  key 
points  and  please  stay  within  the  twenty  minutes  that  you're  allotted.  Conferees  as  a 
whole  please  participate.  I  will  have  to  ask  you  to  hold  your  questions  and  comments  until 
we  have  had  a  series  of  presentations.  Then  we'll  have  a  question  and  answer  period. 
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DPESO  PERSPECTIVE  OF  PRRs 


W.W.  Jeffers 

Chief,  Product  Engineering  Division 
DoD  Product  Engineering  Services  Office 


Three  years  ago,  the  issuance  of  DoD  Directive  5000.34,  "Defense  Production 
Management,"  formalized  Department-wide  attention  toward  minimizing  the  risks 
associated  with  transitioning  defense  systems  from  development  into  production.  In  this 
document,  specific  production  management  responsibilities  were  assigned.  Most  of  you 
attending  this  conference  have  been  deeply  involved  in  fulfilling  the  Military  Departments' 
responsibilities  for  conducting  formal  Production  Readiness  Reviews  (PRRs).  Among  the 
production  management  responsibilities  of  the  Under  Secretary  of  Defense  for  Research 
and  Engineering  is  the  exercise  of  policy  and  operational  control  of  the  DoD  Product 
Engineering  Services  Office  (DPESO)  in  the  conduct  of  its  mission  which  includes: 

•  Providing  production  management  assistance  to  DoD  components. 

•  Providing  independent  assessments  of  producibility  and  production  readiness  of 
major  programs. 

In  this  portion  of  today's  program,  I  will  share  with  you  some  of  the  views  that  our 
office  has  regarding  various  aspects  of  PRRs.  These  views  are  derived  from  our 
observations  and  experiences  in  the  course  of  accomplishing  the  mission  elements  I  just 
cited. 


It  was  originally  intended  that  my  remarks  be  followed  by  Mr.  Gary  Dillard's 
presentation  of  the  results  of  a  recent  study  he  made  to  determine  the  costs  and  benefits 
of  PRRs.  Gary  recently  accepted  a  position  outside  of  our  organization  and  commitments 
there  prevented  his  attendance  here.  Fortunately,  the  findings  from  Gary's  study  serve 
very  nicely  to  clarify  the  rationale  behind  many  of  the  DPESO  views  of  PRRs.  As  a 
consequence  of  these  circumstances,  I  am  going  to  incorporate  most  of  the  elements  of 
this  presentation  into  mine.  Gary  has  concurred  in  this  approach.  The  full  text  of  his 
paper  will  be  included  in  the  conference  proceedings. 

Incidentally,  the  findings  of  the  PRR  Cost  and  Benefits  study  were  derived  from 
interviews  with  14  major  program  offices  and  members  of  the  Production  Engineering 
Division  staff  at  a  major  command. 

•  Objective  of  PRR 

•  DoD  Instruction  5000.38  states:  "The  objective  of  a  PRR  is  to  verify  that  the 
production  design,  planning  and  associated  preparations  for  a  system  have  progressed  to 
the  point  where  a  production  commitment  can  be  made  without  incurring  unacceptable 
risks  of  breaching  thresholds  of  schedule,  performance,  cost,  or  other  established 
criteria." 


In  addition  to  "verify,"  could  include  "help  assure." 

Not  be  used  to  usurp  established  lines  of  program  management  authority 
of  either  the  government  or  the  contractor. 

Overall  goal  -  problem-free  transition  into  production. 
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•  Benefits  of  PRR 


•  Stimulates  in-depth  probing 

•  Provides  early  warning 

•  Worth  the  effort 

•  Contractual  Coverage  for  PRR  Support 

•  Essential  to  insure  that  there  is  legal  basis  for  expenditure  of  contractor  effort 
for  PRR  support 

•  Customarily  addressed  in  SOW 

•  Be  specific  -  avoid  generalities 

Minimizes  subsequent  conflicts  of  understanding 
Minimizes  contractors'  "risk"  pricing 

•  Require  flow  down  to  major/critical  subcontractors 

•  Appropriate  for  major/critical  GFE  contracts 

•  Cost  of  contractor  support 

Consists  of  (1)  manhour  cost  of  preparing  for  and  participating  in  PRR, 
(2)  cost  of  reproducing  data,  (3)  travel  expenses  of  prime  contractor's 
PRR  visits  to  subcontractors 

Has  ranged  from  .02  percent  to  .12  percent  of  program  development  cost 
(.05  percent  average) 

Ratio  of  contractor's  time  for  preparation  and  participation  to  govern¬ 
ment  PRR  onsite  time  has  ranged  from  two  or  five  to  one 
Length  of  a  single  PRR  visit  usually  two  to  five  days 

•  Planning 

•  Initiate  review  effort  early  in  FSED 

Early  identification  of  concerns  facilitates  timely  attention  to  effective 
resolutions 

Reduces  subsequent  PRR  workload 

•  Incremental  reviews 

Avoids  pitfalls  of  a  single  snapshot 

Properly  staffed  and  managed,  can  permit  more  effective  utilization  of 
limited  personnel  resources 

Base  number  and  duration  of  PRR  visits  on  perceptions  of  need  developed 
in  collaboration  with  PMO,  PRR  team  manager,  and  contractor(s) 

Number  and  duration  of  visits  governed  by  complexity  of  program, 
contractor's  demonstrated  responsiveness  to  PRRs,  and  observed  status 
of  production  readiness 

Maximize  cost  effectiveness  of  PRR  by  scheduling  visits  in  logical  geo¬ 
graphic  sequence 

•  Team  staffing 

-  Independence  (freedom  from  parochial  interests)  necessary  for  objec¬ 
tivity  (especially  true  for  design 'status  team  leader) 
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Strive  to  balance  unique  program  experience  (PMO  membership)  with 
objectivity  and  special  expertise  (outside  membership) 

Tailor  selection  of  expertise  to  meet  needs 
Avoid  overloading  with  people 

—  Size  of  team  directly  influences  contractor  manpower  loading  and 
cost  (Ratio  of  1.5-2  contractor  to  1  government  is  typical. 
Government  teams  have  range  from  4-5  to  40-60) 

The  larger  the  team,  the  more  disruptive  to  contractor's  other 
efforts  on  program 

Two  people  often  sufficient  to  cover  each  category  of  the  review 
Excessive  duration  and  membership  of  PRRs  for  other  programs 
have  caused  contractors  to  be  less  responsive  on  subsequent  PRRs 

•  Continuity  of  team  membership  desirable 

"Corporate  memory" 

Uniformity  of  approach  and  consistency  of  evaluation 
Established  rapport  with  contractor  and  fellow  team  members 

•  Resident  CAS  representation  particularly  desirable 

•  Shortage  of  experienced  PRR  personnel 

•  Expanding  availability  of  appropriate  PRR  team  personnel  possible  through: 

Exchange  loan  of  PMO  personnel  to  other  program  PRRs 
Creation  of  roving  cadre  at  command  level 
Utilization  of  independent  consultants 


•  Orientation 


•  Familiarization  of  team  members  (by  PMO,  CAS,  PRR  manager,  etc.) 

Program  under  review  (including  history) 

PRR  schedule,  modus  operandi,  objectives,  etc. 

Principal  issues/concerns  and  past  problems 

Availability  of  relevant  data,  reports,  studies,  and  track  records  of 
contractors 

Respective  responsibilities  of  each  principal  contractor 
Development  and  dissemination  of  industry  outlines  (questions)  relevant 
to  contractors  to  be  visited  which  expand  the  topic  areas  given  in  DoD 
Instruction  5000.38  guidelines 

Native  culture,  traditions,  and  policies  if  PRR  visit  is  foreign-based 

•  Familiarization  of  contractor(s)  (by  PRR  manager,  PMO,  CAS,  etc.) 

Objectives,  schedule,  modus  operandi 
PRR  team  composition 

Clarification  of  scope  and  depth  of  areas  to  be  reviewed  as  well  as 
probable  extent  of  validating  evidence  required 
Provide  inquiry  outlines 

Caliber  of  contractor  personnel  desired  for  interface  with  PRR  team 
•  On-Site  Procedures 

•  Full  gathering  of  all  PRR  participants  (government  and  contractor)  at  onset  of 
each  visit  to: 
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Reiterate  PRR  objectives 

Introduce  participants  and  their  areas  of  PRR  responsibility 
Describe  modus  operandi 

Receive  contractor's  overview  statement  of  program  status,  chief 
concerns,  and  efforts  remaining  (in-house  and  with  subcontractors) 

•  Daily  meeting  of  all  government  participants  to: 

Summarize  findings  in  each  sub  team's  topic  area 
Describe  problems  encountered 

Cross  feed  data  acquired  or  investigative  leads  outside  area  of  primary 
responsibility 

•  Philosophy  of  inquiry 

Use  inquiry  outlines  (questions)  in  open-ended  fashion  -  to  lead  into  and 
stimulate  a  greater  depth  inquiry 

DoD  Instruction  5000.38  guidelines  are  basically  statements  of  conditions 
which  represent  a  satisfactory  production  readiness  posture 
PRR  seeks  factual  evidence  from  which  objective  conclusions  of  status 
can  be  developed 

"Burden  of  proof"  rests  principally  with  contractor(s) 

Status  is  only  as  conclusive  as  the  evidence  which  portrays  it 

Intent  should  be  to  collect,  not  generate,  relevant  data  purely  for  benefit 

of  PRR 

Refrain  from  requiring  non-contractual  data  to  be  in  a  specific  format 
(PRR  team  can  assemble  raw  data  into  usable  form) 

Utilize  relevant  reports  and  studies  rather  than  making  new  roads  to  the 
same  destinations 

Recognize  that  performance,  records  on  past  and  other  ongoing  produc¬ 
tion  efforts  are  rich  sources  of  relevant  data 

•  Do  no  risk  compromising  objectivity  of  PRR  by  intermixing  other  program 
business  with  the  PRR  effort 

•  Out-briefing  with  contractor  highly  desirable 

•  Areas  of  Investigation  Frequently  Underemphasized 

•  Stability  of  design 

Unsettled  design  signals  probability  of  retrofit  activity,  disruptions  in 
production,  schedule  slippages,  cost  growth,  etc. 

Overall  history  of  design  change  rates  more  meaningful  than  recent 
segment  of  history 

Design  stability  remains  questionable  pending  acceptable  test  results 

•  Production  cost 

Cost  now  ranks  as  principal  issue  at  production  decision  point  for  most 
programs 

Substantiate  that  creditability  of  current  design-to-cost  projections  have 
been  established 

Ascertain  the  extent  of  emphasis  placed  on  applying  cost  limiting 
techniques  and  the  effectiveness  of  these  applications 
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•  Long  lead  materials  and  components 

Verify  lead  time  assumptions  for  long  lead  items 

Verify  that  procurement  planning  for  long  lead  items  will  satisfy 
demands  of  production  schedule 

Verify  that  Defense  Priority  Rating  flows  down  to  lower  tier  suppliers 

•  Computer  software 

Just  as  critical  as  system  hardware 

Seldom  attracts  as  much  notice  as  the  hardware 

Most  software  readiness  considerations  are  similar  to  those  for  hardware, 
i.e.  compatibility  between  scheduled  events  and  production  requirements, 
stability  and  level  of  change  activity,  acceptability  of  configuration 
documentation  and  management 

Unique  readiness  considerations  include:  acceptability  of  language 
utilized,  extent  that  reserve  memory  capacity  and  computational  time 
can  accommodate  future  growth 

•  Control  procedures  and  practices 

Documented  policy  statements  and  procedures  only  show  "what" 

PRRs  also  need  to  determine  "how  well"  through  spot  check  of 
procedural  compliance  and  assessments  of  effectiveness  of  applications 

In  closing,  I  want  to  stress  that  the  PRR  process  must  be  sufficiently  flexible  to 
permit  effective  application  in  various  circumstances.  With  this  in  mind,  it  is  my  earnest 
hope  that  the  views  I  have  shared  with  you  will  serve  to  draw  out  additional  views,  ideas, 
and  discussions  in  the  course  of  this  conference. 
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STUDY  OF  THE  BENEFITS  AND  COSTS  OF 
PRODUCTION  READINESS  REVIEWS 


Executive  Summary 


At  the  request  of  the  Office  of  the  Under  Secretary  of  Defense  for  Research  and  Engi¬ 
neering,  the  DoD  Product  Engineering  Services  Office  (DPESO)  surveyed  a  sample  of 
program  management  offices  (PMOs)  to  determine  the  cost  of  contractor  support  for 
their  Production  Readiness  Reviews  (PRRs)  and  the  Program  Managers'  (PMs)  impressions 
on  the  value  of  having  PRRs.  DPESO  found  that  the  cost  of  contractor  support  for  those 
PRRs  was  a  very  modest  percentage  of  the  program's  development  cost.  Of  the 
programs  sampled,  the  PRR  cost  ranged  from  $54. 3K  to  $858K;  the  average  cost  was 
$321. 7K.  The  ratio  of  contractor  cost  for  PRR  support  to  development  cost  was  less  than 
one  percent  and  ranged  from  .02  percent  to  .12  percent  with  an  average  of  .05  percent. 
Generally,  the  PMOs  sampled  are  in  favor  of  PRRs  and  believe  the  benefits  they  received 
were  worth  the  cost.  Two  disadvantages  of  the  PRR  were  cited,  but  there  was  no 
general  consensus  regarding  disadvantages. 

From  the  comments  received  during  the  interviews,  additional  guidance  for  the  conduct 
of  PRRs  was  developed.  The  study  findings  and  recommendations  are  as  follows: 


FINDINGS 


RECOMMENDATIONS 


1.  DoDI  5000.38  lacks  PRR  criteria 
in  the  areas  of  software  and  safety. 


1.  Guidance  (contained  in  text)  should 
be  included  in  future  PRRs,  and  should 
be  included  in  next  revision  of  DoDI 
5000.38. 


2.  PMOs  conducting  foreign  PRRs 
have  problems  not  encountered  on 
domestic  PRRs. 


2.a  Requirements  for  PRR  should  be 
included  in  statement  of  work  (SOW) 
to  prime  and  flowed  down  to 
subcontractors. 

2.b.  Team  members  should  be  well 
versed  in  the  traditions  and  culture 
of  the  foreign  participants. 


3.  Independence  of  the  design  status 
team  leader  is  important  for  an 
objective  PRR. 


3.  A  team  leader  for  the  design 
status  should  not  be  from  the 
developing  office. 


4.  Personnel  experienced  in  PRRs 
are  difficult  to  obtain  for  PRR  teams. 


4.  The  PMO  should  take  every 
opportunity  (such  as  using  another 
program's  PRR)  to  train  personnel. 


5.  The  correct  team  member 
mix  of  PMO  personnel  and  "outsiders" 
is  not  the  same  for  every  program. 


5.  The  PM  must  tradeoff  the 
program  experience  and  expertise 
of  PMO  personnel  with  the  objectivity 
and  lack  of  program  involvement  of 
personnel  from  outside  the  PMO. 
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6.  Large  PRR  teams  and  lengthy 
visits  are  drains  on  the  contractor's 
resources. 

7.  Most  programs  cannot  conduct 
a  PRR  with  single  visits  to  the  prime 
contractor  and  major  subcontractors. 


6.  The  PM  should  tradeoff  the  cost 

of  conducting  the  PRR  with  the  effective¬ 
ness  of  the  team. 

7. a.  Incremental  PRRs  should  be  used 
by  the  PM. 

7.b.  The  number  of  visits  should  be 
based  on  the  nature  of  the  program,  the 
contractor's  prior  experience  with  PRRs 
and  the  government's  prior  experience 
with  the  contractor(s). 


8.  Some  PMOs  are  having  difficulty  8.  Every  program  that  is  going  to 

getting  the  contractor's  full  cooperation  have  PRRs  should  have  a  requirement 

on  PRRs  when  there  is  no  requirement  for  the  PRR  in  the  contract, 

for  PRRs  in  the  contract. 
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STUDY  OF  THE  BENEFITS  AND  COSTS  OF 
PRODUCTION  READINESS  REVIEWS 


I.  Introduction  and  Purpose 

The  Office  of  the  Under  Secretary  of  Defense  for  Research  and  Engineering 
(Acquisition  Policy)  asked  the  DoD  Product  Engineering  Services  Office  (DPESO)  to 
conduct  a  study  to  determine  (1)  the  cost  of  contractor  support  for  a  PRR  and  (2) 
program  managers'  (PMs)  impressions  on  the  benefits,  disadvantages,  and  overall  value  of 
the  Pr oduction  Readiness  Review  (PRR).  As  a  result  of  the  interviews  conducted  for  this 
study,  several  recommendations  for  improving  future  PRRs  were  developed. 

Section  III  of  this  report  contains  the  detailed  findings  for  each  of  the  following 
topics: 

o  Cost  of  contractors  support  for  a  PRR 
o  PMO's  impressions  of  the  PRR 

o  PRR  policy  guidance 

o  Availability  of  PRR  team  personnel 

o  Qualifications  of  PRR  team  personnel 
o  Team  size 

o  Length  of  PRR  visits 

o  Number  of  PRR  visits 

o  Contract  clauses  for  PRR  support 

Section  I'f  presents  the  recommendations  developed  from  these  findings  (attachment  1  is 
a  summary  of  the  study  findings  and  recommendations). 

II.  Methodology 

Sample  programs  were  chosen  based  on  an  estimate  of  the  availability  of  required 
data.  Interviewed  for  this  study  were  the  PMO  staffs  for  the  F-16,  F-18,  EF-111,  and  AV- 
8B  Aircraft;  Precision  Location  Strike  System  (PLSS);  C-5  Wing  Modification;  XM-1  Tank; 
Infantry  Fighting  Vehicle;  Copperhead  Guided  Projectile;  Black  Hawk  and  Advanced 
Attack  Helicopters;  PATRIOT,  HARM,  and  Cruise  Missiles.  Members  of  the  Production 
Engineering  Division  staff  at  the  Army  Missile  Command  (MICOM),  Redstone  Arsenal, 
were  also  interviewed.  Only  Government  personnel  were  interviewed;  no  contractor 
personnel  were  contacted. 

Each  PMO  was  asked  to  provide  an  estimate  developed  within  the  PMO,  of  the 
incremental  cost  of  contractor  support  for  a  PRR.  This  estimate  was  not  to  include  the 
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other  usual  costs  to  prepare  for  production  (i.e.,  the  cost  that  would  normally  occur  if 
there  were  no  PRR).  The  remainder  of  the  information  was  gathered  by  conducting 
interviews  with  senior  PMO  staff  personnel  familiar  with  the  PRR  for  their  program. 

III.  Study  Findings 

A.  Cost  of  Contractor  Support  for  the  PRR 

1.  To  determine  the  cost  incurred,  each  PMO  estimated  the  incremental 
cost  to  the  contractor  for  supporting  the  PRR.  In  all  cases,  the  PMOs  said  the 
incremental  cost  was  the  sum  of:  (1)  the  manhour  cost  for  preparing  and  conducting  the 
review,  (2)  reproduction  cost  of  data  furnished  to  the  PRR  team,  and  (3)  travel  cost  of 
prime  contractor  personnel  traveling  to  subcontractor  PRRs,  if  any.  The  incremental 
cost  for  the  sample  programs  ranged  from  $54. 3K  to  $858K  (Table  1)  with  an  average  cost 
of  $321.7K. 


TABLE  1 

RANGE  OF  COST  ESTIMATES  AND  PERCENTAGES 
FOUND  IN  SAMPLE  PROGRAMS 


PROGRAM 

PRR  COST  AS  A 
PERCENTAGE  OF 
DEVELOPMENT  COST 

PP.R  COST 
ESTIMATE 

SIGNIFICANCE 

A 

.02% 

S390K 

LOWEST 

PERCENTAGE 

B 

- 

$54. 3K 

LOWEST 

PRR  COST 

C 

.12% 

S782.6K 

HIGHEST 

PERCENTAGE 

D 

.04% 

$858K 

HIGHFST 

PRR  CGST 

.05% 

- 

AVERAGE 
PERCENTAGE  j 

- 

$321. 7 K 

AVERAGE 

COST 

2.  The  PRR  cost  was  found  to  be  a  very  modest  proportion  of  the  develop¬ 
ment  .cost.  The  collected  data  was  used  to  estimate  the  ratio  of  PRR  cost  to  program 
development  cost.  This  ratio  was  small,  ranging  from  .0002  to  .0012  for  the  weapon 
systems  sampled  with  an  average  of  .0005.  The  ratio  of  contractor  cost  to  development 
cost  appears  to  be  the  most  useful  guide  to  use  in  estimating  the  magnitude  of  the 
contractor's  cost  of  a  PRR  because  preparation  for  production  is  a  part  of  the 
development  activity. 

3.  The  small  sample  size  precludes  establishing  a  valid  statistical 
correlation  between  the  cost  of  contractor  support  of  a  PRR  and  program  cost. 
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B.  PMO's  Impressions  of  the  PRR 


1.  Those  interviewed  generally  favored  PRRs.  Some  said  they  would  not 
go  into  production  without  one.  Everyone  interviewed  believes  the  PRRs  are  worth  the 
investment  of  time  and  effort.  One  PMO  that  did  not  have  a  PRR  commented  that  a 
PRR  might  have  helped  the  PMO  and  contractor  improve  their  production  planning  and 
eliminate  later  problems. 

2.  All  the  PMO  personnel  interviewed  said  the  PRR  helps  avoid  later  cost 
and  schedule  problems  by  identifying  them  early  in  the  production  planning  phases  of 
full-scale  development.  Some  said  the  PRR  encourages  the  PMO  to  thoroughly  examine 
elements  of  the  production  plan  that  might  otherwise  have  been  overlooked.  As  a  result 
of  one  PRR,  the  affected  contractor  wrote  a  production  plan  for  the  first  time.  The 
PRR  frequently  benefits  other  programs  as  well;  the  contractor's  corrective  actions  are 
often  implemented  plant  wide. 

3.  The  PRRs  at  foreign  subcontractor  plants  benefited  the  PMO  and 
contractors  in  other  ways.  The  PRR  teams  provided  consultant  service  and  aided 
technology  transfer  by  identifying  deficiencies  in  the  subcontractor  production  plan.  The 
PRR  also  highlighted  the  differences  in  business  practices  between  domestic  and  foreign 
defense  contractors. 

4.  Most  of  those  interviewed  cited  the  advantages  above;  few  could  cite 
any  disadvantages.  Two  disadvantages  mentioned  (each  comment  was  made  by  two 
PMOs)  were: 


a.  Contractors  did  not  always  cooperate  fully  with  the  PRR  teams. 
Lack  of  cooperation  was  based  on  contractor  complaints  that  previous  PRR  visits  on 
other  programs  were  too  lengthy  and  had  too  many  PRR  team  members. 

b.  PRR  objectives  changed.  Individuals  from  higher  commands  were 
using  the  PRR  to  direct  the  PM's  and  contractor's  management  of  the  program,  rather 
than  as  a  management  tool  to  assess  program  production  status. 

C.  PRR  Policy  Guidance 

1.  The  majority  of  those  interviewed  said  the  DoD  Policy  Instruction  (DoDI 
5000.38,  "Production  Readiness  Reviews")  is  adequate  and  they  had  no  suggestions  for 
improvement.  Others  suggested  improved  coverage  for:  (1)  software,  (2)  foreign  PRRs, 
(3)  safety,  and  (4)  selection  of  the  group  leader  for  the  program  design  status  review 
team. 


a.  It  was  found  that  PRR  criteria  in  the  area  of  software  had  been 
developed  for  use  on  a  few  PRRs.  Staff  members  in  DPESO  refined  these  criteria  for 
this  paper  (see  section  IV)  as  guidance  for  the  PMOs. 

b.  Lessons  learned  by  two  programs  provided  the  basis  for  guidance 
in  the  area  of  foreign  PRRs.  Those  experienced  with  PRRs  at  foreign  contractor's 
facilities,  found  differences  in  culture  or  language  which  compounded  the  problems 
usually  encountered  on  domestic  PRRs  (even  English-speakng  countries  have  different 
technical  jargon  and  approaches).  The  form  of  government,  Ministry  of  Defense 
structure,  industry/government  relationship,  and  labor/management  relationship,  are  a 


16 


few  of  the  differences  that  must  be  addressed  during  planning  and  execution  of  the  PRR. 
It  was  found  that  flowing  down  the  requirement  for  a  PRR  from  the  prime  contractor  to 
major  subcontractors  is  especially  critical  when  foreign  subcontractors  are  used  or 
contemplated. 

c.  The  criteria  and  guidance  on  safety  and  the  design  status  review 
group  leader  in  section  IV  were  developed  through  interviews  with  PM O  and  DPESO 
personnel. 

2.  All  of  those  interviewed  felt  there  is  sufficient  implementing  guidance 
for  PRRs.  Of  the  three  Military  Departments,  the  Air  Force  has  the  most  guidance  in 
this  area  with  an  Air  Force  Systems  Command  (AFSC)  Guide  for  Manufacturing  Reviews 
(S4_4),  an  AFSC  Regulation  on  PRRs  (84-2),  and  AFSC  Manual  (84-3)  on  Production 
Management.  The  Army  has  issued  Army  Regulation  Number  70-67  (Dec  79)  on  PRRs. 
The  Navy  has  recently  (Apr  80)  issued  Secretary  of  Navy  Instruction  (SECNAVINST) 
4801.IA. 

D.  Availability  of  PRR  Team  Personnel 

Almost  all  those  interviewed  said  civilian  and  military  personnel  experienced 
in  PRRs  were  difficult  to  find.  There  are  few  experienced  military  production  personnel. 
Military  officers  who  are  transferred  out  of  production  oriented  billets  are  not  likely  to 
return.  This  rotation  reduces  the  corporate  memory.  The  problem  of  finding 
appropriate  civilian  personnel  may  lesser  as  more  programs  have  PRRs  and  more  people 
become  familiar  with  the  PRR  procedures.  However,  the  problem  of  obtaining 
adequately  trained  military  personnel  remains  acute  and  is  not  expected  to  improve  in 
the  foreseeable  future. 

E.  Qualifications  of  PRR  Team  Personnel 

There  was  a  substantial  difference  of  opinion  among  the  people  interviewed 
regarding  the  qualifications  for  personnel  being  assigned  to  a  PRR  team.  Some  PMs 
preferred  program  experience  as  a  requirement  for  team  membership.  Others  preferred 
people  from  outside  the  PMO.  The  PMs  recognized  that  the  independence  of  team 
members  is  very  important.  Most  preferred  a  mix  of  PMO  personnel  and  "outsiders." 
The  precise  mix  of  program  people  and  "outsiders"  depended  upon  the  complexity  of  the 
program  and  the  personnel  available  to  the  PM. 

F.  Team  Size 


The  PMOs’  preference  for  the  size  of  the  team  to  be  sent  to  the  contractor 
varied  greatly  also.  Some  said  that  large  teams  (40-60  members)  and  lengthy  visits  by 
previous  PRR  teams  from  other  programs  made  the  contractors  reluctant  to  support 
visits  for  subsequent  PRRs  on  later  programs.  Many  PMs  seemed  to  prefer  small  teams 
(4-5  people).  Others  stated  that  a  large  team  was  necessary  to  explore  all  aspects  of  the 
contractor's  PRR  production  plan. 

G.  Length  of  PRR  Visits 

Some  of  those  interviewed  said  lengthy  visits  were  as  formidable  a  problem  as 
large  teams;  both  were  drains  of  the  contractor's  resources. 
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H.  Number  of  PRR  Visits 


Most  of  those  interviewed  believe  incremental  reviews  are  preferable  to  a 
single  visit.  Incremental  reviews  are  more  costly  but  no  one  interviewed  felt  a  team  of 
reasonable  size  could  assess  the  contractor's  production  readiness  in  just  one  visit.  One 
PMO  commented  the  incremental  reviews  built  up  more  confidence  than  a  single  visit 
could  and  that  the  contractor's  production  planning  was  proceeding  on  schedule. 

I.  Contract  Clauses  for  PRR  Support 

In  the  programs  sampled  for  this  survey,  Air  Force  contracts  usually  included 
detailed  PRR  support  requirements  in  the  statement  of  work  (SOW)  while  the  Navy 
included  PRR  support  requirement  in  contracts  less  often.  Some  of  those  interviewed 
believe  everything  the  PMO  expects  in  the  way  of  contractor  PRR  support  has  to  be 
included  in  the  contract  (SOW).  Others  believe  the  PRR  is  just  another  design  or 
management  review  that  the  contractor  must  support  anyway;  therefore,  PRR  support 
requirements  are  not  needed  in  the  contract.  The  experience  of  the  PMOs  interviewed 
indicates  that  whether  or  not  the  PRR  requirements  should  be  included  in  the  contract  is 
dependent  upon  the  relationship  between  the  PMO  and  the  contractor.  Some  contractors 
provided  support  willingly  without  any  need  for  including  PRR  support  in  the  SOW;  others 
would  only  do  what  was  included  in  the  SOW  and  funded. 

IV.  Conclusions  and  Recommendations 

A.  PRR  Policy  Guidance 

The  cost  data  gathered  indicates  that  the  cost  of  contractor  support  of  a  PRR 
is  a  small  (in  relation  to  the  total  development  cost)  and  a  worthwhile  investment.  Those 
interviewed  had  stated  they  believed  that  the  PRR  was  necessary  and  beneficial,  and 
that  the  benefits  outweigh  the  cost  of  the  PRR.  Program  managers  organizing  a  PRR  for 
their  program  should  take  advantage  of  the  lessons  learned  by  others,  especially  in  the 
areas  discussed  below. 

1*  The  PM  should  include  the  following  criteria  in  his  program  PRR 
guidelines.  In  addition,  the  next  revisions  to  PRR  policy  documents  (such  as  DoDI 
5000.38)  should  include  criteria  for: 


a.  Embedded  Computer  Software 

ti)  Plans  for  programming,  debugging,  testing,  and  integrating 
the  software  with  the  hardware  are  compatible  with  the  production  delivery  schedule. 

(2)  Programming,  debugging,  testing,  and  integration  of  soft¬ 
ware  with  hardware  is  progressing  according  to  approved  plans. 

(3)  Software  change  activity  has  stabilized  at  a  low  level. 

.  .  M  Software  documentation  and  support  facilities  are  adequate 

for  maintenance. 


(5)  A  High  Order  Language,  approved  in  DoD  Instruction 
5000.31,  has  been  used  or  a  waiver  has  been  granted. 
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planned. 


(6)  Appropriate  use  of  standard  computer  hardware  has  been 


(7)  There  is  an  adequate  configuration  management  system  for 
the  modification  of  software  during  production. 

(8)  There  is  adequate  reserve  computer  memory  and  computa¬ 
tional  time  to  accommodate  reasonable  future  growth. 

(9)  Final  acceptance  procedures  for  hardware  and  software  have 

been  identified. 

b.  Safety 

(1)  System  safety  requirements  are  in  conformance  with 

accepted  standards. 

(2)  Plant  layout  and  the  production  plan  are  in  conformance 
with  DoD  and  the  applicable  Military  Department  safety  requirements. 

2.  Each  PM  organizing  a  PRR  effort  should  consider  the  following: 

a.  Design  Status  Review  Group  Leader 

The  PM  should  choose  an  individual  as  the  team  leader  for  the 
design  status  review  who  is  independent  of  the  developing  office.  Choosing  an 
independent  design  status  team  leader  provides  a  "fresh"  look  at  the  design.  The 
importance  of  the  independence  of  team  members  is  discussed  elsewhere  but  it  is  felt 
that  the  independence  in  the  design  status  review  group  is  especially  important. 

b.  Foreign  PRRs 

(1)  Team  members  should  be  well  versed  in  the  countries' 
traditions  and  culture  (perhaps  through  State  Department  indoctrination).  This  would 
lessen  the  possibility  of  conflict  and  misunderstandings  thereby  increasing  the  team's 
efficiency. 


(2)  PMs  should  require  contractor  and  major  subcontractor 
support  for  the  PRR.  Mil-Std  1528  (Production  Management)  should  be  considered 
because  it  requires  that  the  prime  contractor  impose  PRR  responsibilities  on  subcontrac¬ 
tors  if  the  procuring  activity  advises  that  a  PRR  is  to  be  accomplished  at  the  prime. 
Contract  requirements  for  PRRs  (either  by  a  SOW  requirement  or  by  invoking  Mil-Std 
1528)  are  especially  important  when  there  are  foreign  subcontractors. 

B.  PRR  Objectives 


Each  PRR  Government  team  member  should  be  aware  of  the  objective  of  a 
PRR;  the  objective  is  to  assess  the  risks  of  proceeding  into  production.  The  PRR  team 
has  no  authority  to  direct  the  PM  or  contractor.  The  PRR  team  analyzes,  evaluates,  and 
advises  but  does  not  direct. 
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C.  Availability  of  PRR  Team  Personnel 


One  way  to  ease  the  shortage  of  experienced  personnel  is  to  provide  on-the- 
job  training  by  supplying  key  civilian  and  military  personnel  (such  as  group  leaders)  from 
the  PMO  to  other  programs'  PRRs.  A  second  possible  solution  is  to  create  a  cadre  of 
experienced  people  in  the  command  to  provide  assistance  to  the  PMOs.  Another  way  to 
obtain  experienced  people  is  to  have  experienced  consultants  such  as  retired  production 
people. 


D.  Qualifications  of  PRR  Team  Members 

The  composition  of  the  team  must  be  tailored  to  the  program.  The  PMO  must 
trade  off  the  program  experience  and  expertise  of  PMO  personnel  with  the  independence, 
objectivity,  and  lack  of  involvement  of  outsiders.  A  complex  program  may  require  a 
greater  percentage  of  PMO  people  than  a  simple  one.  Another  consideration  is  the 
amount  of  time  personnel  will  spend  away  from  their  normal  duties  to  support  the  PRR. 
The  PMO  staff  can  be  made  available  to  the  PRR  more  easily.  Sometimes,  however, 
specialized  knowledge  is  only  available  from  outside  the  PMO. 

E.  Team  Size 


The  PM  must  tailor  the  size  of  the  PRR  teams  according  to  the 
characteristics  of  his  program.  During  this  study  it  became  apparent  that  the  cost  of 
contractor  support  was  directly  influenced  by  the  size  of  the  Government  team.  The 
data  collected  indicated  the  ratio  of  the  number  of  contractor  personnel  to  Government 
personnel  involved  in  the  PRRs  sampled  ranged  from  1.5  to  2.0  contractor  people  per 
Government  team  member.  This  emphasizes  the  importance  of  the  PM  evaluating  the 
cost  of  conducting  the  PRR  versus  the  effectiveness  of  the  team.  However,  if  possible 
the  PMO  should  have  at  least  two  people  review  each  technical  area  (quality,  design, 
facilities,  etc.)  in  order  to  reduce  biases  and  increase  the  probability  the  deficiencies  will 
be  identified.  For  smaller  teams  this  may  require  each  team  to  cover  more  than  one 
area. 


F.  Length  of  PRR  Visits 

For  the  programs  sampled,  the  ratio  of  contractor's  time  (preparing  for  and 
participating  in  the  review)  to  actual  Government  review  time  ranged  from  2.0  to  5.4. 
For  this  reason  tailoring  the  duration  of  PRR  visits  is  important. 

G.  Number  of  PRR  Visits 


The  number  of  visits  to  each  contractor  should  be  based  upon  the  nature  of 
the  weapon  system,  the  contractor's  previous  experience  with  PRRs,  the  Government's 
prior  experience  with  a  particular  contractor's  products,  and  the  observed  status  of 
production  readiness. 

H.  Contract  Clauses  for  PRR  Support 

Including  PRR  support  requirements  in  the  contract  SOW  appears  to  be  the 
more  prudent  strategy  to  reduce  the  possibility  of  uncooperative  attitudes  by 
contractors.  Attachment  2  gives  examples  of  contract  SOWs  for  PRRs.  The  amount  of 
detail  that  the.  sampled  programs  included  varied  greatly.  However,  the  PMO  cannot  be 
sure  of  receiving  adequate  support  if  it  is  not  required  by  the  SOW.  Therefore,  include 
in  the  SOW  all  of  the  details  necessary  for  contractor  support  at  the  PRR. 
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SUMMARY  OF  STUDY  FINDINGS  AND  RECOMMENDATIONS 


FINDINGS 

1.  DoDI  5000.38  lacks  PRR  criteria 
in  the  areas  of  software  and  safety. 


2.  PMOs  conducting  foreign  PRRs 
have  problems  not  encountered  on 
domestic  PRRs. 


3.  Independence  of  the  design  status 
team  leader  is  important  for  an 
objective  PRR. 

4.  Personnel  experienced  in  PRRs 
are  difficult  to  obtain  for  PRR  teams. 


5.  The  correct  team  member 
mix  of  PMO  personnel  and  "outsiders" 
is  not  the  same  for  every  program. 


6.  Large  PRR  teams  and  lengthy 
visits  are  drains  on  the  contractor's 
resources. 

7.  Most  programs  cannot  conduct 
a  PRR  with  single  visits  to  the  prime 
contractor  and  major  subcontractors. 


8.  Some  PMOs  are  having  difficulty 
getting  the  contractor's  full  cooperation 
on  PRRs  when  there  is  no  requirement 
for  PRRs  in  the  contract. 


RECOMMENDATIONS 

1.  Guidance  (contained  in  text)  should 
be  included  in  future  PRRs,  and 

should  be  included  in  next  revision 
of  DoDI  5000.38. 

2. a.  Requirements  for  PRR  should  be 
included  in  SOW  to  prime  and  flowed 
down  to  subcontractors. 

2. b.  Team  members  should  be  well 
versed  in  traditions  and  culture  of 
the  foreign  participants. 

3.  A  team  leader  for  the  design 
status  should  not  be  from  the 
developing  office. 

4.  The  PMO  should  take  every 
opportunity  (such  as  using  another 
program's  PRR)  to  train  personnel. 

5.  The  PM  must  trade  off  the 
program  experience  and  expertise 

of  PMO  personnel  with  the  objectivity 
and  lack  of  involvement  of  personnel 
from  outside  the  PMO. 

6.  The  PM  should  trade  off  the  cost 

of  conducting  the  PRR  with  the  effective¬ 
ness  of  the  team. 

7.  a.  Incremental  PRRs  should  be  used 
by  the  PM. 

7. b.  The  number  of  visits  should  be 
based  on  the  nature  of  the  program,  the 
contractor's  prior  experience  with 
PRRs,  and  the  Government's  experience 
with  the  contractor. 

8.  Every  program  that  is  going  to 
have  PRRs  should  have  a  requirement 
for  the  PRR  in  the  contract. 


Attachment  1 
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EXAMPLE  NO.  1 


G1O620.O5.O4  Production  Readiness  Reviews  The  Contractor  shall 
support  four  incremental  Production  Readiness  Reviews  (PRRs) . 

The  PRRs  will  be  approximately  five  days  duration  occurring  in 
the  time  frame  of  this  contract.  The  government  will  conduct 
the  reviews  in  sufficient  depth  and  detail  to  determine  the 
Contractors  status  of  production  readiness  and  that  known  or 
anticipated  manufacturing  problems  and  high  risk  areas  have  been 
identified  and  an  impact  assessment  provided.  The  basic  subject 
involved  in  the  PRRs  include  the  following:  Program  and  Manufac¬ 
turing  Schedules,  Make-or-Buy  Plan,  Material/Subcontracting, 
Quality  Assurance,  Industrial  Facilities,  Tooling,  and  Special 
Test  Equipment,  Manpower,  Manufacturing  Methods/Producibility , 
Manufacturing  Management  and  Control  System.  The  team  sizes 
will  be  tailored  as  is  necessary  to  support  the  required  review 
scope  and  depth  (TBD) .  Particular  review  emphasis  shall  depend 
upon:  areas  of  concern  identified  during  the  initial  PRRs; 

progress  in  production  planning  tasks;  manufacturing  progress 
made  by  the  Contractor  in  performance  of  development  and  produc¬ 
tion  long  lead  tasks;  plans  and  progress  toward  transitioning 
from  development  to  initial  low  rate  production,  and  plans  for 
achieving  the  programmed  production  rate.  AFSCR  84-2,  "Produc- 
tion  Readiness  Review,"  will  be  used  as  a  guide  for  preparing 
for  the  support  of  the  government  PRRs.  The  PMO  will  work 
jointly  with  the  Contractor's  PRR  focal  point/manager  to  tailor 
the  requirements  of  AFSCR  84-2  to  meet  the  needs  of  the  Program. 
The  Contractor  shall  be  responsible  for  conducting  additional 
PRRs  at  selected  subcontractors ' /vendors '  plants  that  are 
mutually  agreed  to  by  both  government  and  Contractor.  The 
government  will  participate  as  observers  with  the  Contractor 
during  these  reviews.  The  Contractor  shall  provide  personnel 
and  plant  facilities  necessary  to  support  the  scheduled  reviews. 
Contractor  personnel  who  are  knowledgeable  of  the  management 
anddetail  of  the  production  functions  and  tasks  will  be  made 
available  to  work  with  government  personnel  throughout  the 
period  of  each  review  (DI-E-3118/M) . 
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EXAMPLE  NO.  2 


9.18 

9.18.1 


9.18.2 

9.18.3 


PRODUCTION  READINESS  REVIEW  (PRR) 

The  Contractor  shall  plan,  coordinate,  and  participate 
in  a  PRR  to  demonstrate  production  readiness  prior  to 
start  of  the  production  phase  of  the  program.  The 
areas  to  be  evaluated  in  the  PRR  include  the  following: 

o  Program  and  Manufacturing  Schedules 

o  Subcontract  Management 

o  Make  or  Buy  Program 

o  Material  Selection  and  Procurement 

o  Quality  Assurance 

o  Industrial  Facilities 

o  Tooling 

o  Special  Test  Equipment 
o  Manpower 

o  Manufacturing  Methods/Producibility 

o  Manufacturing  Management  and  Control  System 

The  Contractor  shall  plan  for  a  PRR  to  be  conducted  by 
an  Air  Force  Team  at  the  Contractor's  facility.  The 
PRR  shall  be  accomplished  in  approximately  eight  days. 


The  Contractor  shall  conduct  one  PRR  at  (subcontractor's) 
plant  to  determine  the  production  readiness  of  the  sub¬ 
contractor.  The  areas  to  be  evaluated  will  be  the  same 
as  those  covered  in  the  Contractor's  PRR  and  shall  be 
accomplished  in  approximately  three  days. 


23 


Attachment  2 


EXAMPLE  NO.  3 


12.2.16  Production  Management 

The  contractor  shall  plan  for,  establish,  implement,  and  control 
production  objectives  and  requirements  throughout  this  program 
in  accordance  with  MIL-STD-1528 .  Detailed  planning  for  produc¬ 
tion  shall  commence  as  firm  designs  are  established.  The  con¬ 
tractor  shall  identify,  plan  for,  and  document  production  and 
tooling  test  and  demonstration  requirements  generating  from 
feasibility  assessments  and  producibility  analyses.  All  produc¬ 
tion  requirements  that  require  operations  not  previously  used  in 
the  manufacturing  facility  shall  be  identified  and  plans  to 
introduce  new  operations  in  the  manufacturing  facility  shall  be 
afforded  the  Systems  Program  Office.  The  contractor  shall  pro¬ 
vide  the  necessary  personnel  and  facilities  for,  and  participate 
in,  a  SPO-conducted  Production  Readiness  Review  (PRR)  as  set 
forth  in  AFSCR  84-2.  Approximately  six  preliminary/incremental 
PRRs  will  be  conducted  at  the  prime  contractor  and/or  major 
subcontractor  facilities.  The  final  PRR  will  determine  whether: 
(1)  the  system  under  development  is  ready  for  efficient  and 
economical  production,  (2)  all  critical  production  engineering 
problems  encountered  during  development  have  been  resolved,  and 
(3)  the  contractor  has  accomplished  adequate  planning  for  the 
production  phase.  The  reviews  will  be  of  sufficient  depth  and 
detail  to  adequately  determine  the  contractor’s  status  of  produc¬ 
tion  readiness,  that  known  or  anticipated  manufacturing  problems 
and  high  risk  areas  have  been  identified,  and  an  impact  assess¬ 
ment  provided.  Production  readiness  will  be  evaluated  against 
specific  criteria  established  by  the  Air  Force.  The  criteria 
set  forth  in  AFSCR  84-2  shall  be  considered  to  be  a  representative 
sample.  DI-P-3460 
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BASELINE  INDICATORS  OF  PRODUCTION  READINESS 


John  C.  Bemis 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Stetion,  Alexandria,  VA  22314 


ABSTRACT 


This  paper  presents  findings  resulting  from  e  pre¬ 
liminary  study  of  the  feasibility  of  establishing 
quantifiable  baseline  indicators  for  production 
reediness  of  defense  systems  prior  to  the  produc¬ 
tion  decision  milestone.  Eleven  production  read¬ 
iness  indicetore  ere  discussed  end  the  character¬ 
istics  of  eech,  based  on  empirical  data  from  pre¬ 
vious  systems,  ere  examined. 

Preliminary  findings  of  this  study  indicate  that 
various  elements  of  production  readiness  oan  be 
measured.  Additionally,  it  has  been  determined 
that  while  eech  system  is  unique,  there  exiets  a 
number  of  common  characteristics  which  can  be  com¬ 
pared  on  a  system-by-system  basis.  The  following 
list  of  baseline  indicators  was  established: 

HARDWARE  INDICATORS 

1.  Engineering  Change  Traffic  Profiles 

2.  Reliability  Growth  Patterns 

3.  Yield  Rates  for  Special  Manufacturing  Processes 

4.  Yield  Rates  for  Test  Operations 

5.  Scrap  and  Rework  Levels 

6.  Level  of  Effort  on  Nonconforming  Materials 

7.  Out-of-Stetion  Work  Performed 

SOFTWARE  INDICATORS 

1.  Rate  of  Discovery  of  Errors 

2.  Rate  of  Change  of  Requlremente 

3.  Rate  of  Change  of  Revision  Level 

4 .  Percent  Memory  and  Speed  Capacity  Unconnltted 

Projected  profilee  for  the  behavior  of  these  indi¬ 
cators  during  the  development  phase  of  programs  are 
shown,  end  the  implication  of  each  indicator  is 
discussed 

INTRODUCTION 

Evaluating  the  production  readiness  of  e  weapon 
eystem  prior  to  the  production  deoision  point  ie  an 
important  element  of  the  DoD  weapon  eystem  acquisi¬ 
tion  process.  Production  readiness  is  assessed  by 
means  of  e  Production  Readiness  Review  (PRR).  De¬ 
partment  of  Defense  Instruction  5000.38,  titled 
"Production  Readiness  Reviews,"  defines  the  purpoat 
of  e  PRR  as  follows:  "The  objective  of  e  PRR  ie  to 
verify  that  the  production  design,  planning,  and 
associated  preparations  for  a  system  have  progress¬ 
ed  to  the  point  where  a  production  commitment  can 
be  made  without  incurring  unacceptable  rieks  of 
breaching  thresholds  of  eohedule,  performance, 


cost,  or  other  esteblished  criteria."  The  criteria 
used  to  establish  production  readiness  has  consist¬ 
ed  of  a  series  of  largely  subjective  Judgements, 
with  guidance  provided  by  checklists.  The  Product 
Engineering  Services  Office  undertook  e  study  to 
establish  meaningful  and  meesurable  indicators  of 
production  readiness  based  on  data  that  ie  normally 
required  as  part  of  defense  contrects.  This  paper 
discueses  the  preliminary  findings  of  the  study, 
which  esteblished  e  list  of  eleven  potential  indi¬ 
cators  and  examined  methods  for  quantifying  them. 

In  selecting  the  indicetors,  the  following  fectors 
were  considered: 

1.  It  was  recognized  that  each  system  was  made  up 
of  a  number  of  fectors  whioh  made  e  partioular 
system  unique  from  any  other  eystem  in  terms  of  ite 
problems,  time  ecales,  state-of-the-art  applica¬ 
tions,  budget  restraints,  end  the  neoessity  for 
ohenges  during  development. 

2.  Date  for  the  individual  indicetore  ehould  be 
eveileble  either  in  the  Program  Office  or  et  the 
contractor's  facility  es  e  result  of  normally 
imposed  contractual  requirements.  No  requirements 
for  new  data  items  were  oontempleted. 

3.  The  indicetors  should  be  simple  in  oonoept  and 
readily  understood. 

4.  Since  numerical  values  for  the  indicetors  are 
variable  in  time,  trend  date  was  ooneidered  to  be 
of  more  velue  than  point  data. 

Based  on  these  considerations,  the  following  poten¬ 
tial  baseline  indicetors  were  examined: 

HARDWARE  INDICATORS 

1.  Engineering  Change  Trefflc 

2.  Reliebilillty  Growth  Petteme 

3.  Yield  Retes  for  Specie!  Manufacturing  Prooesses 

4.  Yield  Retes  for  Test  Operations 

5.  Scrap  end  Rework  Levels 

6.  Level  of  Effort  on  Nonconforming  Materiel s 

7.  Out-of -Stetion  Work  Performed. 

SOFTWARE  INDICATORS 

1.  Rate  of  Discovery  of  Errors 

2.  Rate  of  Change  of  Requirements 

3.  Rate  of  Chenge  of  Revision  Level 

4.  %  Memory  and  Speed  Cepaclty  Unoommitted 

Fol owing  is  e  discussion  ~of  each  indicator,  ite 
potentiel  program  implications,  and  a  projeoted 
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profile  of  the  indicator  for  the  development  phase 
of  a  program. 

DISCUSSION 

Engineering  Change  Traffic  Profiles.  Examination 
of  the  profile  of  engineering  change  traffic  can  be 
revealing  in  terms  of  the  design  maturity  of  a 
system,  as  well  as  providing  symptoms  of  specific 
problems  in  the  areas  of  fabrication,  inspection 
and  test  operations,  subcontracted  equipment,  or  in 
system  specifications.  When  the  number  of  engine¬ 
ering  change  made  is  plotted  against  a  time  scale 
which  includes  the  development  cycle,  a  pattern 
such  as  that  shown  in  figure  1  will  normally  occur. 


PROTOTYPE  FABRICATION  _  TESTING 


FIGURE  1.  PROJECTED  PROFILE  FOR  ENGINEERING  CHANGE  TRAFFIC 

The  number  of  changes  starts  at  a  zero  point  prior 
to  the  engineering  release.  As  hardware  fabrica¬ 
tion  is  initiated,  the  number  of  changes  increases 
to  a  maximum  and  should  decline  as  engineering 
problems  are  resolved.  At  the  completion  of  the 
prototype  build,  the  number  of  changes  should  have 
followed  a  downward  trend  to  a  reasonable  level. 
During  prototype  testing  an  increase  in  changes  is 
noticed  due  to  problems  detected  during  the  tests. 
The  curve  depicting  the  number  of  changes  versus 
time  should,  again,  follow  a  downward  trend  to  a 
reasonable  level.  Sustained  levels  of  high  change 
rate  indicate  a  risk  to  cost,  schedule,  and/or 
performance.  The  appearance  of  an  apparent  exces¬ 
sive  number  of  changes  at  the  completion  of  the 
prototype  build  should  raise  questions  as  to  cause. 
It  is  obvious  that  both  cost  and  schedule  con¬ 
straints  would  be  extreme-y  difficult  to  meet  with 
an  extended  period  of  high  change  rate.  Empirical 
data  compiled  during  this  study  indicated  that  the 
shape  of  the  engineering  change  traffic  profile  was 
of  a  similar  shape  for  different  kinds  of  systems 
including  aircraft,  electronic  systems,  tracked 
vehicles,  and  gun  systems.  The  profile  is  suf¬ 
ficiently  defined  such  that .anomalies  can  be  iden¬ 
tified  and  investigated.  The  cause  and  effect 
relationship  between  engineering  changes  and 
acquisition  cost  growth  is  well  understood,  but 
remains  to  be  quantified.  This  quantification  will 
be  investigated  during  Phase  II  of  the  study. 

Reliability  Growth  Patterns.  A  considerable  amount 
of  data  has  been  developed  relative  to  reliability 
growth  which  results  from  finding  and  fixing  reli¬ 
ability  problems  during  testing.  When  this  growth 
is  plotted  to  linear  scales  the  resulting  curve 


appears  as  an  exponential.  Large  gains  in  reli¬ 
ability  are  made  initially  as  the  "easy  to  fix" 
problems  are  corrected.  As  the  reliability  in¬ 
creases,  gains  in  reliability  become  more  difficult 
to  obtain.  The  result  is  that  the  demonstrated 
reliability  approaches  a  limit  on  an  asymptotic 
basis.  When  this  data  is  transformed  to  a  log/log 
format,  the  reliability  growth  curve  becomes  a 
straight  line,  which  provides  a  convenient  method 
for  projecting  probable  future  values  for  reli¬ 
ability.  The  horizontal  axis  represents  the  evalu¬ 
ation  exposure  in  terms  of  hours  tested,  miles 
driven,  rounds  fired,  or  other  testing  criteria 
suitable  to  the  system.  The  vertical  axis  repre¬ 
sents  the  reliability  measure  in  terms  of  mean  time 
between  failures,  mean  miles  between  failure,  suc¬ 
cess  rate,  etc.  Experience  with  a  number  of  pro¬ 
jects  has  allowed  the  derivation  of  a  generalized 
set  of  criteria  for  these  curves  during  the  devel¬ 
opment  phase  of  a  program.  Figure  2  depicts  the 
projected  profile  for  reliability  growth  and  indi¬ 
cates  the  following  guidelines  when  considering 
reliability  growth  during  the  development  phase: 


FIGURE  2.  PROJECTED  PROFllEFOR  RELIABILITY  GROWTH 


1.  As  released,  at  least  10  percent  of  the  required 
reliability  should  be  realized  during  first  tests. 

2.  A  reliability  prediction  based  on  a  mathemati¬ 
cal  model  of  the  system  would  preferably  indicate  a 
reliability  of  125  percent  of  the  required  reli¬ 
ability. 

3.  The  growth  curve  should  have  a  slope  of  at  least 
0.1  and  is  unlikely  to  exceed  a  slope  of  0.6. 

4.  Lack  of  promising  reliability  test  results 
during  development  testing  or  lack  of  definite 
pattern  of  reliability  growth  during  subsequent 
testing  can  be  caused  by  a  number  of  factors  in¬ 
cluding  marginal  design,  parts  or  materials 
problems,  and  ineffective  corrective  action.  The 
inability  to  meet  reliability  requirements  during 
development  represent  a  risk  to  both  acquisition 
cost  and  schedule.  Inability  to  meet  reliability 
requirements  in  a  fielded  system  can  have  a  major 
impact  on  operating  and  support  costs.  The  trend 
of  the  reliability  growth  pattern  for  a  system 
provides  valuable  insights  into  production  read¬ 
iness. 
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Yl^ld  Rates  for  Special  Manufacturing  Processes.  A 

significant  problem  in  meeting  cost  and  schedule 
can  result  from  low  yield  rates  for  special  manu¬ 
facturing  processes.  Also,  it  would  be  unusual  to 
find  a  new  weapon  system  in  which  one  or  more 
"state-of-the-art"  manufacturing  processes  was  not 
employed.  Under  the  sponsorship  of  contractor  R&D 
programs,  and  under  the  sponsorship  of  the  DoD 
Manufacturing  Technology  Program,  a  significant 
number  of  special  processes  are  under  development 
at  any  given  time.  This  constant  development  is 
necessary  in  order  to  reduce  costs,  increase  per¬ 
formance,  increase  productivity,  and  to  advance  the 
technology  base.  Examples  of  these  special 
processes  in  the  electronics  field  include  methods 
for  producing  large  scale  integration  (LSI)  de¬ 
vices,  multi-layer  printed  circuit  boards,  and  high 
density  memory  devices.  Examples  of  these  process¬ 
es  in  the  mechanical  area  include  the  laser  machin¬ 
ing  and  Joining,  inertia  welding,  electro-chemical 
and  electrical  discharge  machining,  vacuum  plasma 
coating,  and  advanced  methods  for  non-destructive 
testing. 

Low  yield  rates  for  any  of  the  special  processes 
would  have  adverse  effects  on  program  cost  and/or 
schedule  and,  therefore,  could  represent  a  program 
risk.  The  projected  profile  for  special  process 
yield  rates  is  shown  in  figure  3.  As  a  new  process 


ACCEPTABLE  PRODUCTION  YIELDS 


FIGURE  3.  PROJECTED  PROFILE  FOR  TESWROCESS  YIELDS 


is  developed,  the  initial  yield  rates  will  be  lower 
than  the  ultimate  yield  rates  as  the  process  vari¬ 
ables  are  being  defined  and  controlled.  Normally, 
the  major  process  variables  are  controlled  first, 
leading  to  significant  gains  In  yield  rates.  In 
the  later  stages  of  development,  the  "fine  tuning" 
of  the  process  takes  place.  The  "fine  tuning" 
generally  leads  to  smaller  gains  and  yield.  Be¬ 
cause  of  this,  the  process  yield  approaches  an 
ultimate  yield  value  asymptotically. 

A  plot  of  process  yield  rate  versus  time  should 
have  the  following  characteristics: 

1.  A  significant  growth  should  be  evident  in  the 
yield  rate  as  a  function  of  time  or  units  process¬ 
ed. 


2.  The  yield  rate  attained  during  the  latest 
period  should  be  acceptable  in  terms  of  dollar 
risk.  It  is  difficult  to  place  limits  on  the  yield 
rate  from  a  special  process  since  the  economic 
consequences  represent  a  wide  variation  from  one 
process  to  another.  Restated,  the  cost  of  a  reject 
from  one  process  may  represent  a  few  cents  while  a 
reject  from  another  process  may  represent  thousands 
of  dollars. 

It  is  anticipated  that  the  yield  rates  for  special 
processes  will  lend  themselves  to  a  transform  into 
a  straight  line.  Data  is  being  collected  to  indi¬ 
cate  what  "normal"  yield  rate  values  may  be  expect¬ 
ed  from  these  special  processes. 

Yield  Rates  for  Test  Operations.  Examining  the 
yield  rates  for  test  operations  and  the  trends  in 
these  rates  can  be  helpful  in  the  identification  of 
a  number  of  different  kinds  of  problems.  Low  yield 
rates  can  result  from  marginal  designs/specifica¬ 
tions,  or  difficulties  with  parts,  materials,  or 
processes.  Since  both  acquisition  cost  and  sche¬ 
dule  are  sensitive  to  the  yield  rates  for  test 
operations,  it  is  important  that  these  rates  attain 
satisfactory  levels.  Figure  3  depicts  the  project¬ 
ed  profile  for  growth  in  yield  rates  for  test 
operations.  Sustained  period  of  low  yield  rates 
without  evidence  of  improvement  would  indicate  the 
presence  of  chronic  problems  which  either  have  not 
or  cannot  be  solved  through  the  management  systems 
responsible  for  corrective  action.  As  in  the  case 
of  yield  rates  for  special  processes,  it  is  antici¬ 
pated  that  yield  rates  for  test  operations  will 
lend  themselves  to  a  suitable  data  transform  such 
that  straight  line  projections  of  yield  rates  can 
be  made. 

Scrap  and  Rework  Levels.  Excessive  scrap  and  re¬ 
work  can  add  significantly  to  acquisition  cost  and 
be  detrimental  to  schedule.  If  a  contractor’s 
corrective  action  system  is  working  effectively, 
experience  has  shown  that  the  rates  for  sorap  and 
rework  should  be  decreasing  as  a  function  of  time 
and  production  of  additional  units.  The  most  ap¬ 
propriate  model  for  the  behavior  of  scrap  and  re¬ 
work  rates  appears  to  be  the  standard  learning 
curve.  Figure  4  is  the  projected  profile  for  these 
curves  in  situations  where  effective  corrective 
action  is  taking  place. 


PROTOTYPE  FABRICATION  ^ 


FIGURE  4.  PROJECTED  PROFILE  FOR  SCRAP/REWORK 
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Attributes  of  these  carves  should  be  an  acceptable 
improvement  slope  as  well  as  a  reasonable  absolute 
value  at  the  completion  of  the  development  models. 
Since  scrap  and  rework  can  be  accounted  for  by 
several  different  methods,  care  should  be  taken  to 
assure  that  the  same  denominator  is  used  in  each 
percentage  being  compared.  For  example,  common 
methods  of  calculating  rework  value  are  as  a  per¬ 
centage  of  direct  production  labor  and  as  a  per¬ 
centage  of  rework  dollars  to  product  dollars. 

Excessive  scrap  and/or  rework  can  serve  as  a  symp¬ 
tom  of  a  wide  variety  of  problems  in  both  the  design 
and  manufacturing  areas.  Lack  of  improvement  in 
scrap  and  rework  values  is  an  indicator  of 
deficiencies  in  the  management  systems  responsible 
for  corrective  action. 

Nonconforming  Materials.  Since  there  are  a  large 
number  of  causey  for  nonconforming  material,  and 
they  are  all  indicative  cf  problems,  the  level  of 
effort  being  expended  in  the  area  of  nonconforming 
material  can  provide  valuable  insight  into  the 
overall  health  of  a  program.  Additionally,  observ¬ 
ing  the  trend  of  nonconforming  material  indicators 
can  provide  insight  into  the  effectiveness  of  the 
management  systems  that  are  designed  to  prevent 
nonconformances  and  to  provide  corrective  action. 
Some  of  the  methods  used  for  tracking  non-conform¬ 
ing  materials  include  the.  following. 

1.  Plotting  the  gross  number  of  nonconforming 
material  reports  written  against  successive  end 
items. 

2.  Calculating  the  labor  dollars  involved  in  mak¬ 
ing  disposition  of  nonconforming  materials  through 
the  material  review  process. 

3.  Calculating  reject  rates  as  percentages. 

It  appears  that  regardless  of  the  specific  method 
used  for  tracking  nonconformances,  a  decreasing 
trend  should  occur  as  shown  in  figure  *4.  Steady 
progress  in  lowering  the  nonconformance  rate  and  a 
reasonable  level  for  the  latest  result  should  be 
evident..  The  collection  of  data  from  additional 
programs  may  reveal  that  the.  slope  of  this  line 
could  be  used  as  a  measure  of  the  effectiveness  of  a 
contractor’s  corrective  action  system. 

Out-of-Station  Work  Performed.  This  indicator  is 
primarily  oriented  toward  large  systems  where  the 
development  models  are  assembled  on  a  production 
line  as  opposed  to  being  assembled  in  an  engineer¬ 
ing  laboratory  (e.g.  aircraft).  There  are  a  number 
of  reasons  for  performing  production  work  out-of- 
station,  and  essentially  all  of  them  are  indicative 
of  problems.  These  causes  include  late  delivery 
from  subcontractors,  tooling  problems,  design  pro¬ 
blems,  shortage  of  skilled  labor  in  specific  cat¬ 
egories,  etc.  As  shown  in  figure  5,  the  percentage 
or  absolute  measurement  of  out-of-station  work 
would  be  expected  to  increase  after  the  initiation 
of  assembly  work.  After  reaching  a  peak  value,  the 
rate  should  start  to  decrease  as  prototype  units 
are  being  completed  and  problem  areas  are  being 
corrected.  Continued  high  levels  of  out-of-station 
work  would  indicate  that  the  problems  causing  the 
out-of-station  work  are  not  being  solved.  Such  a 
continuation  of  high  rates  for  out-of-station  work 
would  represent  a  risk  to  the  program  in  terms  of 


acquisition  cost  and/or  schedule  slip.  Ideally, 
the  completion  of  the  prototype  units  would  see 
out-of-station  work  at  a  reasonable  rate  after 
having  followed  a  downward  curve  from  previously 
higher  levels. 


FIGURE  5.  PROJECTED  PROFILE  FOR  OUT  OF  STATION  WORK 


Baseline  Software  Indicators.  The  number  of  DoD 
procurements  involving  computer  software  is  rising 
and  this  trend  is  expected  to  continue.  Studies  on 
this  subject  have  indicated  that  by  1980,  software 
will  account  for  approximately  80  percent  of  the 
computer  related  dollars,  with  hardware  accounting 
for  the  other  20  percent.  The  adequacy  of  the 
software  is  vital  to  the  performance  of  the  weapon 
system.  In  recent  years  considerable  effort  has 
been  expended  in  developing  metrics  for  determining 
the  condition  of  software  development  programs. 
Some  of  these  metrics  have  proved  to  be  highly 
complex,  and  in  some  cases  impractical.  Only  re¬ 
cently  have  practical  and  readily  understandable 
measures  of  software  development  begun  to  appear. 
Underscoring  the  importance  of  determining  the 
condition  of  the  software  development  effort  is  the 
fact  that  this  effort  is  essentially  completed 
prior  to  the  production  release  for  hardware. 

For  the  purpose  of  this  report,  four  measures  of 
software  readiness  are  suggested: 

•  Rate  of  Discovery  of  Errors 

•  Rate  of  Change  of  Requirements 

•  Rate  of  Change  of  Revision  Level 

•  Percentage  of  Memory  and  Speed  Capacity  Uncom¬ 

mitted  - - 

From  experience  on  a  number  of  software  development 
programs,  the  rate  of  discovery  of  errors  appears 
to  follow  a  predictable  pattern.  Starting  with 
coding  checks  and  proceeding  into  each  successive 
test  phase,  the  rate  of  discovery  of  errors  starts 
out  at  a  relatively  high  level  and  follows  a  down¬ 
ward  slope  as  problems  are  corrected,  "starting 
with  system  integration  testing,  the  initial  peak 
rate  for  error  discovery  decreases.  These  rela¬ 
tionships  are  shown  in  figure  6. 
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FIGURE  6.  PROJECTED  PROFILE  FOR  REQUIREMENTS  CHANGES. 
ERROR  DISCOVERY  AND  CODING  REVISIONS 


Because  of  the  ease  of  changing  software,  as  oppos¬ 
ed  to  hardware,  software  changes  are  frequently 
used  to  effect  mission  changes  and  to  correct 
deficiencies  in  other  subsystem  areas.  Excessive 
requirement  changes  in  the  software  can  indicate 
potential  hardware  problems  and  a  lack  of  maturity 
in  the  system  requirements.  Figure  6  depicts  the 
normal  behavior  pattern  for  Engineering  Change 
Orders  issued  against  the  Design  Specification. 
This  experience  curve  is  analogous  to  the  engineer¬ 
ing  traffic  curve  for  hardware,  in  that  it  follows 
a  downward  trend,  and  then  experiences  smaller  and 
smaller  peaks  as  each  successive  level  of  testing 
is  undertaken.  The  net  effect  resembles  a  damped 
sine  wave.  Significant  deviations  from  this  gen¬ 
eral  form  should  be  identified  as  to  cause.  En¬ 
gineering  Change  Orders  issued  against  the  Design 
Specification  cause  the  revision  level  of  the  soft¬ 
ware  to  be  changed,  and  thus  the  rate  of  change  of 
revision  level  can  also  be  used  as  an  indicator  of 
program  development  maturity. 

The  percent  of  memory  and  speed  yet  uncommitted 
appears  to  be  a  promising  indicator,  but  additional 
data  will  have  to  be  gathered  before  this  indicator 
can  be  quantified.  Small  values  for  uncommitted 
speed  and  memory  have  implications  relative  to  the 
ability  to  absorb  future  changes  and  the  difficulty 
4n  performing  all  required  functions  within  the 
remaining  capacity. 


CONCLUSIONS 

Preliminary  findings  of  this  study  indicate  that 
various  elements  of  production  readiness  can  be 
measured.  Additionally,  it  has  been  determined 
that  while  each  system  is  unique,  there  exists  a 
number  of  common  characteristics  which  can  be 
compared  on  a  system-by-system  basis. 

The  first  phase  of  the  baseline  indicators  study 
has  identified  a  list  of  measurable  indicators, 
established  methods  for  their  analysis,  and  result¬ 
ed  in  the  collection  of  a  substantial  amount  of 
empirical  data  from  existing  programs.  The  next 
phase  of  the  study  will  be  devoted  to  establishing 
the  relationship  between  the  initial  values  for  the 
indicators  and  the  program  outcomes  in  terms  of 
acquisition  cost,  schedule,  and  performance. 
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Introduction 


This  paper  addresses  a  problem  often  neglected  during  a  Production  Readiness  Review 
(PRR)  -  determining  the  "production  readiness"  of  the  computer  software  used  by  the 
system.  The  problem  is  largely  the  result  of  the  abstract  nature  of  software. 
Nevertheless,  ways  must  be  found  for  the  PRR  team  to  make  an  accurate  assessment  of 
the  risks  involved  in  going  into  production  with  the  existing  software. 

The  Software  Production  Problem/Software  "Failures" 


At  the  onset  it  is  important  to  state  the  major  distinction,  from  a  PRR  point  of  view, 
between  software  and  hardware.  The  hardware  production  problem  addressed  during  a 
PRR  is  whether  or  not  copies  of  the  prototype  hardware  produced  during  engineering 
development  can  be  effectively  produced  in  the  factory.  The  software  production  problem 
is  quite  different. 

Once  the  first  copy  of  error-free  software  is  available,  producing  duplicate  copies  is 
straight  forward.  The  software  media  may  be  magnetic  tape,  magnetic  disk  or  even 
especially  manufactured  Read  Only  Memories.  No  matter  which  media  is  used, 
duplication  and  verification  of  the  duplication  is  relatively  easy.  The  difficulty  is  in 
producing  the  first  error-free  copy  and  in  testing  it.  Unlike  hardware,  software  does  not 
have  a  Mean-Time  Between-Failure.  It  does  not  "fail"  when  it  "wears  out"  like  hardware 
does.  Software  "fails"  when  an  error  in  the  coding  produces  an  incorrect  response  to  a 
given  input.  The  range  of  incorrect  responses  goes  from  providing  an  incorrect  answer  to 
halting  the  computer.  The  software  errors  were  always  in  the  coding  but  not  recognized 
as  such.  A  simple  software  error,  such  as  a  keypunching  error,  is  usually  easy  to  find  and 
eliminate  because  many  values  of  input  produce  incorrect  responses.  (In  fact,  software 
development  programs  can  often  locate  such  errors  )  This  is  not  the  case  for  complex 
software  errors.  For  example,  a  complex  software  error  might  produce  an  incorrect 
response  only  when  one  system  starts  at  a  particular  point  in  time  relative  to  another;  any 
other  starting  time  produces  the  correct  response.  Since  relatively  few  inputs  (or  maybe 
only  a  single  one)  produce  incorrect  responses,  complex  software  errors  are  extremely 
difficult  to  find. 


Importance  of  Assessing  Software  Status 

Figure  1  shows  the  trend  during  the  last  decade  of  adding  more  and  more  embedded 
computer  hardware  and  software  to  avionics  systems. 
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Most  engineers  are  familiar  with  the  large  decrease  in  computer  hardware  costs  during  the 
last  several  years.  Similar  decreases  have  not  come  about  in  the  software  area  lareelv 
due  to  the  labor  intensity  of  the  software  development  process.  This  results  in§the 

shown'nFigure”  ^  "d  hardware  1:05,5  for  DoD  embedded  computers  during  the  80's 
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The  chart,  taken  from  the  October  27,  1980  issue  of  Electronic  Engineering  Times,  shows 
that  hardware  costs  will  increase  times  while  the  software  costs  will  increase  over  11 
times.  As  software  becomes  a  larger  and  larger  percentage  of  total  acquisition  costs,  it 
becomes  more  and  more  important  for  the  PRR  team  to  assess  the  software  status  during 
their  reviews. 

A  second  reason  software  status  should  be  assessed  during  the  PRR  is  that  software,  like 
hardware,  is  more  expensive  to  correct  in  the  field  than  in  the  factory.  The  third  and 
most  important  reason  for  software  assessment  is  that  inoperable  software  will  result  in  a 
system  that  is  useless  in  the  field.  Not  all  corrections  may  be  made  by  "just  changing 
software."  Some  corrections  may  require  hardware  modifications;  the  need  for  such 
modifications  can  not  be  known  until  the  entire  system  (including  software)  is  adequately 
tested.  The  PRR  team  should  be  extremely  wary  of  any  system  which  is  ready  for  full 
scale  production  except  for  problems  "just  in  the  software." 

Software  Problem  Areas 


There  have  been  several  reccurring  software  problems  that  have  been  present  in  embedded 
computer  systems  developed  in  the  past.  There  is  only  a  limited  amount  of  computer 
memory  available  in  a  given  embedded  computer.  There  is  also  only  a  limited  amount  of 
computer  processing  time  available.  (For  example,  all  processing  of  a  given  set  of  radar 
data  must  be  accomplished  before  the  next  set  of  data  arrives.)  When  memory  or 
computation  time  limits  are  approached,  programmers  must  employ  error-prone  tech¬ 
niques  that  produce  computer  coding  that  is  difficult  to  debug  and  almost  impossible  to 
modify  in  later  years. 

Determination  of  software  development  status  is  very  difficult  and  in  the  past  has  often 
been  left  to  the  programmers  and  analysts  most  directly  involved  in  the  development 
process.  Managers  have  been  provided  reports  on  the  percent  completion  of  the  software. 
Typically  progress  was  good  in  the  early  phases.  However,  at  around  the  90  percent 
completion  point,  many  analysts  realized  system  integration  and  testing  would  be  more 
time  consuming  than  originally  planned.  Also  the  problem  to  be  solved  usually  evolves 
into  something  quite  different  than  the  one  originally  specified.  This  results  in  the  type  of 
curve  shown  in  Figure  3. 


OR 

'/.PROJECT  FUNDS 


Figure  3.  Estimate  of  Software  Status 
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Note  that  the  estimate  of  percent  completion  goes  from  an  optimistic  90  percent  half  way 
through  the  project  to  the  more  realistic  60  percent  at  the  originally-planned  project 
completion  date.  This  "90  percent  completion  problem"  is  largely  the  result  of  applying 
less  engineering  discipline  to  software  than  to  other  areas  during  the  engineering 
development  process. 


Software  Indicators 

Four  software  readiness  indicators  that  should  be  available  during  a  PRR  are  (1)  memory 
utilization,  (2)  available  computation  time,  (3)  wftware  error  discovery  rate,  and  (4) 
Software  Engineering  Change  Order  rate. 

Figure  4  shows  the  anticipated  pattern  of  (a)  error  discovery  rate,  (b)  amount  of  coding 
that  must  be  revised  to  correct  the  errors  and  (c)  requirement  changes. 


Note  that  entering  each  new  phase  produces  bum  s  on  each  of  the  curves  when  new 
classes  of  problems  are  seen  for  the  first  time. 


Figure  5  shows  memory  or  core  allocation  for  an  actue,  embedded  computer  system. 
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The  relatively  straight  line  shows  the  memory  allocation  budgeted  at  the  beginning  of  the 
project.  The  stepped  line  shows  periodic  updates  based  on  actual  memory  allocation. 
Note  that  twenty  percent  of  the  available  memory  was  reserved  for  further  expansion  of 
the  system.  This  memory  reserve  could  be  used  to  modify  the  system  to  accommodate 
different  enemy  threats  or  to  make  the  system  easier  to  use. 

Figure  6  shows  computation  time  allocation  for  the  same  system. 
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(Since  four  computers  are  processing  data  concurrently,  there  is  a  maximum  of  4000 
milliseconds  of  processing  time  available  in  each  real-time  second.)  Note  the  current 
estimate  of  computation  time  exceeds  the  specified  limit.  It  must  be  decided  whether  to 
leave  the  software  as  written  and  accept  a  reduced  reserve  or  to  maintain  the  twenty 
percent  reserve  at  the  expense  of  rewriting  some  of  the  code,  possibly  using  some  of  the 
undesirable  techniques  previously  mentioned. 

Figure  7  shows  monthly  software  error  discovery  rate  as  a  function  of  time. 


TIME  (IN  MONTHS) 

Figure  7.  Software  Errors  Discovered 


The  points  connected  by  the  solid  line  indicate  actual  data  points  while  the  dashed  lines 
shows  the  result  of  the  best  Rayleigh  distribution  fit  to  the  points.  The  trend  is  obviously 
good  and  the  software  seems  under  control. 

Figure  8,  from  the  same  project,  gives  quite  a  different  picture.  (The  time  period  of 
Figure  7  covers  approximately  the  second  half  of  the  time  period  shown  in  Figure  8.) 
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The  profile  of  monthly  software  engineering  change  order  activity  clearly  indicates  that 
the  software  requirements  have  not  stabilized.  Comparison  of  Figures  7  and  8  illustrates 
the  advisability  of  examining  more  than  one  indicator  of  software  maturity. 

If  formal  software  testing  is  required,  the  Mean-Time-Between -Computer -Program-Errors 
can  be  plotted  as  a  function  of  number  of  hours  of  testing. 


Figure  9.  Mean  Time  Between 
Computer  Program  Errors 


In  the  example  in  Figure  9  there  is  a  good  linear  fit  when  the  data  is  plotted  on  log-log 
paper.  Note  the  difference  between  the  curve  for  instantaneous  and  cumulative  values. 
Which  Mean-Time-Between-Computer- Program-Errors  to  use  must  be  carefully  spelled 
out  in  the  testing  procedures  if  this  parameter  determines  whether  the  software  passes  or 
fails  the  formal  software  test. 
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Software  that  is  often  not  Reviewed 


There  are  several  important  software  items  that  are  often  ignored  during  PRRs.  The  first 
is  non-deliverable  software  that  directly  effects  production  of  hardware.  An  example  is 
the  software  used  to  drive  computer-controlled  machines.  Although  there  is  strict 
configuration  management  of  engineering  drawings,  there  may  be  no  configuration 
management  of  the  software  that  determines  what  will  be  produced. 

Development  costs  for  Automatic  Test  Equipment  software  are  often  greater  than  for 
operational  software.  For  example,  the  F-16  program  had  28  operational  computer 
programs  compared  to  600  for  test  and  support.  Many  of  the  test  and  support  programs 
may  have  a  direct  effect  on  factory  production  if,  as  in  the  case  of  the  F-16,  the  test 
equipment  was  to  be  used  for  factory  testing  of  electronic  subsystems. 

There  is  a  large  unseen  base  on  which  the  application  software  depends.  Figure  10  shows 
this  relationship  as  an  iceberg. 
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Figure  10.  The  Software  Iceberg 


A  series  of  incremental  PRR  visits  facilitate  the  discovery  of  inadequacies  in  the 
underwater"  software  items.  Such  items  may  have  a  direct  impact  on  obtaining  the 
operational  software  on  time.  In  addition,  they  form  the  basis  of  support  and  changes 

after  deployment.  Examination  of  these  items  during  PRRs  would  insure  user's  later  needs 
can  be  met. 


DPESO  Software  Projects 

The  DoD  Product  Engineering  Services  Office  (DPESO)  is  involved  with  two  projects 
directly  related  to  reviewing  software  during  PRRs.  In  1977  the  Chairman  of  the  DoD 
Management  Steering  Committee  for  Embedded  Computer  Resources  and  I  developed 
Embedded  Computer  Resources  and  the  DSARC  Process  -  A  Guidebook*."  The  guidebook 
included  questions  that  the  Office  of  the  Secretary  of  Defense  staff  might  ask  the 
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Program  Manager's  staff  prior  to  DSARCs  I,  II,  and  III.  This  guidebook  is  now  being 
revised  and  a  section  on  Production  Issues  is  being  added  to  the  chapter  covering  DSARC 
III. 

The  second  area  being  investigated  is  whether  a  computer  model  can  be  developed  to  help 
a  PRR  team  determine  software  development  status.  This  is  a  joint  study  between  DPESO 
and  the  DoD  Computer  Institute.  DPESO  will  be  providing  actual  contractor  data 
obtained  during  PRR  visits  while  the  Computer  Institute  will  be  providing  the  computer 
modeling  expertise.  This  study  has  just  started  and  will  take  approximately  14  months  to 
complete. 


Conclusion 


This  paper  addressed  the  importance  of  measuring  software  during  the  PRR  and  touched 
on  some  indicators  that  can  be  used  to  access  software  development  status.  This  is  a 
relatively  new  area  for  PRRs  and  much  remains  to  be  learned.  Hopefully  meetings  like 
the  OSD/Joint  Services  Production  Readiness  Reviews  Conference  will  provide  a  forum 
for  exchanging  ideas  on  how  we  can  better  determine  the  production  readiness  of 
computer  software. 

Ideas  on  how  to  assess  software  status  as  well  as  actual  data  are  solicited.  They  should  be 
sent  to  Lee  Schumacher,  DPESO,  c/o  DLA,  Cameron  Station,  Alexandria,  VA  22314. 
Telephone  (703)  756-2335  or  Autovon  289-2335. 


*  The  1977  edition  of  the  guidebook  is  available  through  the  Defense  Technical  Infor¬ 
mation  Center  (formerly  Vs  Defense  Documentation  Center)  or  the  National  Technical 
Information  Service  as  AD  Number  A046398. 
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DLA/OCAS  ROLE  IN  PRODUCTION  READINESS  REVIEWS 
INTRODUCTION 

In  order  to  improve  comnuni cat i ons  and  effectiveness  of  Defense  Contract  Administration 
Services  (DCAS)  support  in  Production  Readiness  Reviews  (PRRs)/  it  is  essential  that 
Program  Managers  be  familiar  with  how  DLA(CAS)  is  organized/  what  functional  capabili¬ 
ties  it  possesses/  what  support  can  be  provided/  and  how  to  acquire  this  support.  The 
objective  of  this  presentation  is  to  provide  a  review  of  the  knowledge  required  to 
insure  a  coordinated  effort  is  achieved  among  the  Program  Management  Offices  and  the 
DCAS  elements  involved  in  accomplishing  PRRs. 

ORGANIZATION 


Headquarters  DL A  is  located  at  Cameron  Station/  Alexandria/  Virginia.  Lt  Gen  Gerald  J. 
Post/  US  A  F /  is  the  Director.  He  has  two  deputies.  One/  MG  Robert  C.  Gaskill/  USA/ 
heads  the  buying  functions  of  D LA .  These  include:  Contracting  Directorate/  Supply 
Operations  Directorate/  Technical  &  Logistics  Services  Directorate/  six  Supply  Centers/ 
four  Si- »5ly  Depots/  and  six  Service  Centers.  The  second  deputy  to  Lt  Gen  Post  is 
r  A  DM  Gerald  J.  Thompson  who  heads  the  Defense  Contract  Administration  Services  functions 
of  HQ  DLA.  Two  executive  directorates  support  R A  DM  Thompson.  The  first/  headed  by 
Mr.  Williar.  V.  Gordon  (SES)  is  the  Executive  Director/  Contract  Management.  There  are 


nine  divisions  under  Mr.  Gordon  (see  attached  chart)  responsible  for  providing  Head¬ 
quarters  policy  and  direction  on  matters  involving  Contract  Management.  The  second 
Executive  Director  under  Admiral  Thompson  is  the  Quality  Assurance  Directorate 
headed  by  R  A  DM  Cruden.  Under  Admiral  Cruden  are  three  divisions  (see  attached  chart) 
responsible  for  providing  policy/  guidance/  and  technical  assistance  on  matters 
involving  contractor  compliance  with  contract  quality  reqt i rements  and  efforts  to 
assure  the  product  delivered  to  the  government  meets  the  speci f i catons  delineated  by 
contract. 

DCAS  is  currently  organized  into  nine  geographical  Regions.  They  are:  Atlanta/  Boston/ 
Chicago/  Cleveland/  DaL.as/  Los  Angeles/  New  York/  Philadelphia/  and  St.  Louis. 

Scattered  within  these  Regions  are  37  Management  Areas  (DCASMAs) /  34  Plant  Representa¬ 
tive  Offices  (DCASPROs)/  and  900  residencies. 

FUNCTIONS 

The  organizational  structure  by  function  at  HQ  DLA  within  DCAS  is  mirrored  in  each  of 
the  field  offices.  The  Systems  &  Engineering  Division/  or  in  the  field/  the  Systems  & 
Engineering  Office/  is  the  Offifce  of  Primary  Responsibility  for  PRRs.  There  are 
approximately  400  engineers  assigned  in  the  field  offices.  In  addition/  within  DCAS 
there  a.e  510  Quality  Assurance  Engineers  and  800  Industrial  Specialists.  The 
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personnel  assigned  to  DCAS  are  highly  trained  and  experienced  professionals.  They 
constitute  a  considerable  body  of  technical  expertise  familiar  with  the  capacity  and 
capability  of  American  Industry.  This  represents  a  substantial  resource  to  Program 
Managers  which  must  be  cultivated  and  can  be  utilized  to  get  the  most  effective  use 
of  our  limited  Defense  Budget. 


PRR  SUPPORT 

Procedures  for  DCAS  support  of  PRRs  are  contained  in  DL AM  8300. 1,  Production  Manual 
for  Contract  Administration  Services.  Other  references  are: 

DoD  Instruction  5000.38/  Production  Readiness  Reviews 
DoD  Directive  5000.34/  Defense  Production  Management 
M0U  between  DLA(CAS)  and  DCS/Systems/  AFSC/  6  Sep  73 
The  Program  Manager  should  call  or  write  the  Commander  of  the  cognizant  Contract 
Administration  Office/  either  the  DC ASM A  or  the  DCASPR0  Commander  of  the  area  in  which 
the  PRR  is  to  be  performed.  Addresses  and  telephone  numbers  are  found  in  DoD  Handbook 
4105.59/  DoD  Directory  of  Contract  Administration  Services  Components.  The  letter 
should  request  that  both  the  Director  of  Contract  Management  and  the  Director  of 
Quality  Assurance  be  alerted  to  the  requirement  for  a  PRR.  A  courtesy  copy  of  this 
letter  should  be  sent  to  the  Commander  of  the  Region  in  which  the  PRR  is  being  performed/ 


and  also  a  copy  should  be  provided  to  HQ  D L A /  ATTN:  DLA-AE/  Cameron  Station/ 
Alexandria/  Virginia/  22314.  In  the  Letter/  identify  the  type  of  support  needed 
and  the  areas  to  be  covered  during  the  PRR.  The  Program  Manager  can  then  expect 
DCAS  representatives  to  be  onsite  and  ready  to  assist  the  PRR  Team  when  it  arrives. 
He  can  expect  an  in-briefing  from  the  assigned  DCAS  engineer  on  the  strengths  and 
weaknesses  of  the  contractor  involved.  He  can  also  expect  that  our  Engineers/ 
Industrial  Specialists  and  Quality  Assurance  Specialists  are  generally  well  equipped 
to  answer  questions  pertaining  to  the  topics  listed  on  the  attached  charts  under 
"PRR  Support."  These  specialists  should  also  be  available  to  provide  surveillance 
and  follow  up  on  any  corrective  actions  required/  and  to  provide  progress  reports 
back  to  the  Program  Manager  at  some  agreed  to  interval. 

The  Systems  &  Engineering  Division  (DLA-AE)  at  HQ  DL A  should  be  notified  any  time  a 
serious  problem  arises  with  regards  to  a  lack  of  support  for  any  PRR  within  DCAS/  or 
if  assistarce  is  needed  in  acquiring  support.  Contact  either  Colonel  Warne  or 
Dr.  Moul  or  AV  28-47132. 
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DLA/DCAS  ROLE  IN  PRRs 


•  ORGANIZATIONAL  STRUCTURE 


•  FUNCTIONAL  CAPABILITIES 


•  SUPPORT  TO  PRR  TEAM 


HQ  DLA  ORGANIZATION 
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HQ  DLA  ORGANIZATION  (DLA-A) 


•OPR  FOR  PRODUCTION  READINESS  REVIEWS  (PRRs) 


HQ  DLA  ORGANIZATION  (DLA-Q) 
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DEFENSE  CONTRACT  ADMINISTRATION  SERVICES  REGION 
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DCAS  SUPPORT  TO  THE  PRR  TEAM 


EVENT 

•  NOTIFICATION  FROM  PROGRAM  MANAGER 

•  ARRIVAL  OF  PROGRAM  MANAGER  TEAM 

•  PLANT  VISIT(S) 

•  IN  BETWEEN /AFTER  PLANT  VISITS 

•  POST  REVIEW 


ACTIONS 

DCAS  ENGINEER  IS  DESIGNATED 
DCAS  SPECIALISTS  ARE  SELECTED 

BRIEFINGS  PROVIDED  BY  DCAS 

ARRANGEMENTS /FACILITIES 

FULL/PART  TIME  SUPPORT  FROM 
SPECIALISTS  AS  NEEDED 

COORDINATION  &  FOLLOWUP 

SURVEILLANCE  OF  ACTION  ITEMS 
PERIODIC  REPORTING 


DCAS  SUPPORT  TO  THE  PRR  TEAM 

•  PLANT  RESOURCES  /  FACILITIES 

•  ADEQUACY  FOR  RATE 

•  TIMELY  ACQUISITION  /  INSTALLATION 

•  AUTOMATED  PRODUCTION  TECHNIQUES 

•  CONTRACTOR  PERSONNEL 

•  MANNING  LEVELS 

•  SKILLS  DEVELOPMENT /TRAINING 

•  CERTIFICATION 

•  PRODUCTION  ENGINEERING  /  PLANNING 

•  SCHEDULE  COMPATIBILITY 

•  COST  REDUCTION 

•  ALTERNATIVE  CAPACITIES 

•  CONFIGURATION  MANAGEMENT 

•  HANDLING  OF  ENGINEERING  CHANGES 

•  MANAGEMENT  INFORMATION  SYSTEM  ADEQUACY 
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DCAS  SUPPORT  TO  THE  PRR  TEAM 


•  MATERIALS  /  PURCHASED  PARTS 

•  LONG  LEAD  ITEMS 

•  PROCUREMENT  PLAN 

•  SELECTION  OF  SUBCONTRACTORS 

•  VISIBILITY  OF  SUBCONTRACTORS 

•  QUALITY  ASSURANCE 

•  INTEGRATION  WITH  PRODUCTION  PLANNING 

•  CORRECTIVE  ACTION 

•  CONTRACT  ADMINISTRATION 

•  ATTITUDES 

•  CHARACTER 

•  WORKING  RELATIONSHIPS 


CONCLUSIONS 


•  ON  SITE  CAPABILITY 

•  FUNCTIONALLY  ORGANIZED 

•  TECHNICALLY  ORIENTED 

•  DEDICATED  TO  SUPPORT  PROGRAM  MANAGERS 
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US  ARMY  'PRODUCTION  READINESS  REVIEW'  OVERVIEW 


PRESENTED  TO:  OSD/JOINT  SERVICES  PRODUCTION 
READINESS  REVIEW  CONFERENCE 


PRESENTED  BY:  FREDERICK  J.  MICHEL 

ACTING  CHIEF,  OFFICt  OF 
MANUFACTURING  TECHNOLOGY 


19  NOVEMBER  1980 


US  ARMY 

"PRODUCTION  READINESS  REVIEW" 
AN  OVERVIEW 


PRESENTED  TO: 

OSD/JOINT  SERVICES  PRODUCTION 
READINESS  REVIEW  CONFERENCE 

PRESENTED  BY:  FREDERICK!  MICHEL 

ACTING  CHIEF,  OFFICE  OF 
MANUFACTURING  TECHNOLOGY 
19  NOVEMBER  1980 
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OUTLINE 


PRR  GUIDANCE 

PRR  RESPONSIBILITIES 

SCOPE  OF  THE  PRR 

CONDUCT  OF  THE  PRR 

CONCLUSION  -  STATUS/LESSONS  LEARNED 


OUTLINE 

GOOD  MORNING. 

DURING  THE  NEXT  FEW  MINUTES  I  WANT  TO  GIVE  YOU  A  SHORT  OVERVIEW  ON  SOME  OF  THE  KEY 
POINTS  OF  PRODUCTION  READINESS  REVIEWS  AS  CONDUCTED  BY  TIE  ARMY. 

•  FIRST  I  WILL  GIVE  YOU  AN  OVERVIEW  OF  THE  PRODUCTION  READINESS  REVIEW, 

IN  TERMS  OF  GUIDANCE  AND  OBJECTIVES. 

•  I  WILL  THEN  GO  INTO  SOME  DETAILS  ABOUT  THE  SCOPE  OF  THE  PRR  AND  WHAT 
CRITERIA  WE  USE  TO  JUDGE  PRODUCTION  READINESS. 

•  I'LL  TALK  ABOUT  THE  CONDUCT  OF  THE  PRR  -  AND  FINALLY, 

•  WHAT  WE'VE  LEARNED  ABOUT  PRR's  BASED  ON  THE  ONES  WE'VE  CONDUCTED  THUS  FAR. 

AS  YOU  CAN  SEE  FROM  THE  OUTLINE  I  AM  GOING  TO  SPEND  ONLY  A  SMALL  AMOUNT  OF 
TIME  ON  THE  REGULATION,  BUT  AM  GOING  TO  CONCENTRATE  ON  THE  PRACTICAL  ASPECTS. 


50 


PRR  GUIDANCE 


Q  REFERENCE:  DODD  5000. 34,  DEFENSE  PRODUCTION  MANAGEMENT 
DODI  5000.38,  PRODUCTION  READINESS  REVIEW 
AR  70-67,  PRODUCTION  READINESS  REVIEW 

e  PRR'S  FOR  ALL  MAJOR/NON-MAJOR  SYSTEMS 

©  PRR  RESULTS  TO  DECISIONMAKERS: 

-  ASARC  II 171 1 IA 

-  DSARC  1 1  I/I  1 1 A 

-  DEVA  I  PR 

-  SPECIAL  PRODUCTION  I  PR'S 

©  ARMY  PRODUCT  ENGINEERING  SERVICES  OFFICE  (APESO) 

®  FOR  MAJOR  SYSTEMS,  INDEPENDENT  DOD  ASSESSMENT  (DPESO) 


PRR  GUIDANCE 

THREE  BASIC  REGULATORY  DOCUMENTS  CONCERNING  PRODUCTION  READINESS  REVIEWS  HAVE 
BEEN  PUBLISHED  OVER  THE  PAST  2  YEARS,  NAMELY  DODD  5000.34,  DODI  5000.38  AND 
AR  70-67.  AR  70-67  IMPLEMENTS  THE  DOD  DIRECTIVES/INSTRUCTIONS  AND  WAS  EFFECTIVE 
1  UANUARY  1980. 

WITHIN  THE  ARMY,  PRR's  ARE  REQUIRED  FOR  ALL  SYSTEMS,  AND  THE  RESPECTIVE  PROJECT 
MANAGERS  ARE  RESPONSIBLE  FOR  HAVING  THEM  CONDUCTED. 

IN  ADDITION  TO  THE  PRR  ITSELF,  THE  ARMY  REQUIRES  AN  INITIAL  PRR  (IPRR)  EARLY  IN 
THE  ENGINEERING  DEVELOPMENT  CONTRACT  AND  NOT  LATER  THAN  12  MONTHS  PRIOR  TO 
MILESTONE  111  DECISIONMAKERS  - 

THE  ARMY  PRODUCT  ENGINEERING  SERVICES  OFFICE  (APESO)  HAS  BEEN  ESTABLISHED 
WITHIN  THE  OFFICE  OF  MANUFACTURING  TECHNOLOGY  AT  DARCOM  TO  ASSIST  IN  THE 
CONDUCT  OF  PRR's  AS  WELL  AS  TO  ESTABLISH  AND  REINFORCE  PRR  POLICIES/PROCEDURES. 
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ORGAftliZATION  CHART 


OFFICE  OF  MANUFACTURING  TECHNOLOGY 

THIS  CHART  SHOWS  THE  POSITION  OF  THE  OFFICE  OF  MANUFACTURING  TECHNOLOGY 
WITHIN  THE  ARMY  STRUCTURE. 
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OFFICE  OF  MANUFACTURING  TECHNOLOGY 


DEPUTY  CG 
FOR 

MATERIEL  DEVELOPMENT 


t - - 

CHIEF 
nccini:  nc 

MFG  TECH 

4 NO LOGY 

AVIATION 


FINANCIAL  MGMT 


TRACKED  VEH 
WEAPONS 


TACTICAL  VEH 
OTHER  SUPPORT 
EQUIPMENT 


COMMUNICATIONS 
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OMT  ORGANIZATION 

THIS  CHART  SHOWS  THE  ORGANIZATION  OF  THE  OFFICE  OF  MANUFACTURING  TECHNOLOGY. 
ALL  PRODUCTION  RELATED  FUNCTIONS  IN  THE  LIFE  CYCLE  OF  A  WEAPONS  SYSTEM  WHICH 
CAN  AFFECT  END  ITEM  COST  ARE  IN  ONE  ORGANIZATION  STARTING  WITH  PRODUCIBILITY 
ENGINEERING  PLANNING  (PEP)  AND  ENDING  WITH  PRR's. 


53 


PRR  RESPONSIBILITIES 


ARMY  PESO 

@  PROVIDE  PRR  CONSULTATION 
©  PARTICIPATE  IN  PRR'S 
0  ASSIST  IN  RECRUITING  TEAM 

•  ASSIST  WITH  ANALYSIS 
e  COORDINATE  WITH  DOD  PESO 
o  PROVIDE  EXAMPLES  OF  PRR  PROCEDURES 
e  ASSIST  IN  PREPARING  PRR  SOW  FOR  RFP 


PRR  RESPONSIBILITIES 

THE  ARMY'S  PRODUCT  ENGINEERING  SERVICES  OFFICE  WAS  CREATED  TO  ESTABLISH  AND 
REINFORCE  PRR  POLICIES/PROCEDURES.  BUT,  FURTHER,  IT  WAS  CREATED  AS  A  SERVICE 
ORGANIZATION.  WE  ASSIST  THE  PM  IN  PLANNING  AND  CONDUCTING  PRR's  IN  ACCORDANCE 
Mil  FHL  REGULATIONS.  IN  ADDITION  WE  MAKE  PRODUCTION  ENGINEERING  TALENT 
AVAILABLE  FOR  PARTICIPATION  ON  THE  PRR  TEAM  TO  HELP  IN  THE  PRODUCTION 
READINESS  ANALYSIS. 
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LIFE  CYCLE  MANAGEMENT  MODEL 


PHASE 

EVENT-^^^ 

ACTIVITY 

CONCEPTUAL 

utroasa3nrgxg.uiL.-j  i  ant— p 

DEMONSTRATION 

& 

VALIDATION 

l  IJH  J :  r.  -mr>?:.:.Tan  i  mu,  ■ 

FULL-SCALE 

DEVELOPMENT 

PRODUCTION 

L 

DEPLOYMENT 

TESTING 

ILS 

PEP 

MMT 

PRR 

FACILITIES 

PRODUCTION 

DEPOT 

|  _ _ 

IerqujesiJ 

Tf/o  EV.  1 

DTI  /  OTI 

DT  II /OT  II 

1 

TC  \ 

> 

1  ~~i 

1  .  ! 

i-  T 

1  _□ 

L _  < 

HARDWARE 

CONFIGURATION 

EREADBOARD  & 
EXPERIMENTAL 

BRASSEOARD  & 

Advanced  Development 

ENGINEERING 

DEVELOPMENT 

-rr,  r  -r.tr  j’Vii’n.'r 

initial  !  Full 

Product!  on  1  Product!  on 

MILESTONES  0  I  II  ||| 


PKR  AND  THE  LIFE  CYCLE 
MANAGEMENT  MODEL 

THE  PRODUCTION  READINESS  REVIEW  IS  AN  INTEGRATED  PART  OF  THE  ARMY'S  PRODUCT  LIFE 
CYCLE. 

FOR  MAJOR  SYSTEMS,  THE  PRR'S  ARE  CONDUCTED  INCREMENTALLY,  COMMENCING  AS  SOON  AS 
POSSIBLE  AFTER  MILESTONE  II  AND  THE  FINAL  PRR  COMPLETED  NLT  2  MONTHS  PRIOR  TO 
DSARC  III  /  ASARC  III  /  DEVAIPR;  TO  ALLOW  ADEQUATE  REVIEW  BY  THE  MILESTONE  III 
DECISION  MAKERS. 
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SCOPE  OF  THE  PRR 


FORMAL  EXAMINATION  OF  A  PROGRAM'S: 

•  PRODUCT  DESIGN 

•  INDUSTRIAL  RESOURCES 

•  PRODUCTION  ENGINEERING/PLANNING 
o  MATERIAL/PURCHASED  PARTS 

•  QUALITY  ASSURANCE 

•  LOGISTICS 

•  CONTRACT  ADMINISTRATION 

•  PROGRAM  MANAGEMENT 


SCOPE  OF  THE  PRR 

NOW  I'D  LIKE  TO  GO  A  LITTLE  DEEPER  INTO  THE  SCOPE  OF  THE  PRR. 

AS  YOU  CAN  SEE  FROM  THIS  SLIDE,  A  PRR  EXAMINES  PRODUCTION  READINESS  ACTIVITIES 
"ACROSS  THE  BOARD".  IT  IS  CONSIDERED  ONE  OF  THE  KEY  MILESTONES  IN  THE  LIFE 
CYCLE  OF  A  WEAPONS  SYSTEMS  FOR  IDENTIFYING  POTENTIAL  MANUFACTURING  PROBLEMS. 
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PRR  CRITERIA 


PRODUCT  DESIGN 


PRODUC I B I LITY 

DESIGN  CHANGE  ACTIVITY 

TDP 

STANDARDIZATION 
CRITICAL/SCARCE  MAT'LS 
PROPRIETARY  MAT'LS/PROCESSES 
PRODUCTION  COST  PROJECTIONS 


INDUSTRIAL  RESOURCES 


FACILITIES 

PRODUCTION  EQUIPMENT 
TEST  EQUIPMENT 
TOOLING 
PERSONNEL 


PRODUCTION  ENGR/PLANNIN 


PRODUCTION  PLAN 
PRODUCTION  SCHEDULES 
MFG.  METHODS 
VALUE  ENGR/COST  REDUCTION 
ALTERNATE  PROD'N  APPROACHES 
PROD'N  DW'GS/STANDARDS/INSTRUCTION 
CONFIGURATION  MANAGEMENT 
ENGR  CHANGES  DURING  PRODUCTION 


PRR  CRITERIA 


HERE  IS  A  SOMEWHAT  MORE  DETAILED  LISTING  OF  THE  CRITERIA  WHICH  ARE  USED  TO 
JUDGE  PRODUCTION  READINESS  PROGRESS. 


WITHIN  PRODUCT  DESI6N, 


•  PRODUCIBIL ITY 

•  DESIGN  CHANGE  ACTIVITY,  INDICATING  MATURITY  OF 
DESIGN. 

•  STATUS  OF  THE  TDP. 

•  WHETHER  THERE  ARE  RISKS  RELATED  TO  USE  OF 
CRITICAL  OR  PROPRIETARY  MATERIALS  AND  PROCESSES. 

•  THE  RISK  IN  NOT  MEETING  CURRENT  COST  PROJECTIONS. 


IN  INDUSTRIAL  RESOURCES,  WE'RE  LOOKING  AT  THE  NUMBERS  AND  CAPABILITIES  OF 

FACILITIES,  EQUIPMENT  AND  TOOLING  REQUIRED  AS  WELL 
AS  PERSONNEL  REQUIRED. 


FOR 


PRODUCTION  ENGINEERING  /  PLANNING, 

•  A  PRODUCTION  PLAN. 

•  INCLUDING  MASTER  SET-BACK  SCHEDULES. 

•  MANUFACTURING  PROCESS  IDENTIFICATION. 

•  SHOP  INSTRUCTIONS;  WORKAROUND  PLANS  FOR  PROBLEM 
AREAS. 


PRR  CRITERIA 


MATERIAL/PURCHASED 

PARTS 


BILL  OF  MATERIAL 
MAKE/BUY 
LONG  LEAD  ITEMS 
SOLE  SOURCE  ITEMS 
GFE 

MAT' L/ INVENTORY  CONTROL 
PROCUREMENT  PLANNING 


QUALITY  ASSURANCE 


QA  ORGANIZATION 
QA  PROCEDURES/CRITERIA 
QUALITY  PROGRAM/PLAN 


LOGISTICS 


SPARES  PRODUCTION 

SPARES  PLANNING/SCHEDULING 

TMDE 

TRAINING 

LOGISTICS  DATA 

TRANSPORTATION 

OVERHAUL  PROCEDURES 

RAM  ADEQUACY 


PRR  CRITERIA 

IN  MATERIALS/PURCHASED  PARTS,  A  PLAN  FOR  ACQUIRING,  SCHEDULING  AND  CONTROLLING 

ALL  MATERIAL  -  -  MAKE  OR  BUY. 

A  QA  PLAN,  WITH  SPECIFIC  QA  PROCEDURES  IDENTIFIED  FOR  THIS  SYSTEM. 

IN  LOGISTICS,  HAS  PLANNING  BEEN  ACCOMPLISHED  FOR  SPARES,  TMDE,  TRAINING 

DEVICES,  DATA,  ETC.  AND  DOES  THE  CAPABILITY  EXIST  TO  PRODUCE 
THESE  ITEMS? 
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PRR  CRITERIA 


CONTRACT 

ADMINISTRATION 


GOVT  ON-SITE  ORGANIZATION 
GOVT  PROCEDURES 

-  ECP'S/ECO'S 

-  PROPERTY  ACCOUNTABILITY 
COST/SCHEDULE  CONTROL  SYSTEM 


PROGRAM 

MANAGEMENT 


MANAGEMENT  STRUCTURE 
MANAGEMENT  CAPABILITIES 
SUBCONTRACT  MGT 
MGT  INFORMATION  SYSTEM 


PRR  CRITERIA 

CONTRACT  ADMINISTRATION  -  THE  INTERACTION  OF  THE  GOVERNMENT  WITH  THE  CONTRACTOR, 
HAVE  PROCEDURES  BEEN  FORMULATED  AS  TO  DCAS's  ROLE  ?  DOES  THE  CONTRACTOR 
HAVE  A  VALIDATED  CS2  SYSTEM  ? 

IN  PROGRAM  MANAGEMENT,  WE  LOOK  AT  THE  MANAGEMENT 

•  STRUCTURE  AND  CAPABILITIES  OF  A  CONTRACTOR  AND  WHETHER  THEY  HAVE 
THE  BACKGROUND  TO  ACCOMPLISH  THE  JOB. 

•  IS  THEIR  MANAGEMENT  INFORMATION  SYSTEM  EFFECTIVE  IN  DEFINING  PROBLEMS 
TO  BE  RESOLVED  IN  PRODUCTION. 


t 
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CONDUCT  OF  THE  PRR 


FORMAL  REVIEWS: 

INITIAL  PRODUCTION  READINESS  REVIEWS  (IPRR) 

•  REQUIRED  FOR  MAJOR  SYSTEMS 

•  FIRST  LOOK  AT  PRODUCTION  READINESS  STATUS 

•  INTRODUCE  CONTRACTORS  TO  PRR' S 

•  COMPLETE  12  MONTHS  BEFORE  MILESTONE  1 1 1 
PRODUCTION  READINESS  REVIEWS  (PRR) 

•  FORMAL,  DOCUMENTED,  SYSTEMATIC  REVIEWS 

•  FINAL  LOOK  AT  PRODUCTION  READINESS 

•  FORMAL  RECOMMENDATION  FOR  MILESTONE  1 1 1 
PRODUCTION  DECISION 

•  COMPLETE  2  MONTHS  BEFORE  MILESTONE  III 


CONDUCT  OF  THE  PRR 

THERE  ARE  TWO  BASIC  TYPES  OF  FORMAL  REVIEWS  IN  THE  ARMY'S  PRR  PROCESS  -  - 

•  THE  INITIAL  PRR,  WHICH  IS  REQUIRED  ONLY  FOR  MAJOR  SYSTEMS,  IS  TO  BE 
COMPLETED  12  MONTHS  BEFORE  MILESTONE  III.  WE  RECOMMEND  THE  IPRR  BE 
CONDUCTED  AS  SOON  AS  POSSIBLE  AFTER  MILESTONE  II  TO  INTRODUCE  THE 
CONTRACTORS  TO  WHAT  IS  EXPECTED  OF  THEM  TO  GET  READY  FOR  PRODUCTION. 

•  THE  PRR's  THEMSELVES  ARE  REQUIRED  TO  BE  COMPLETED  2  MONTHS  PRIOR  TO 
MILESTONE  III.  WE  RECOMMEND  THAT  THE  PRR's  BE  CONDUCTED  ON  AN 
INCREMENTAL  BASIS  THROUGHOUT  ENGINEERING  DEVELOPMENT  WITH  THE  FINAL 

REVIEW  SERVING  AS  THE  BASIS  FOR  THE  RECOMMENDATION  AT  DECISION  POINT 
III. 
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CONDUCT  OF  THE  PRR 


ORGANIZATION /MAN  POWER 
POSSIBLE  TEAM  ORGANIZATION 


CONDUCT  OF  THE  PRR 

THE  USE  OF  FUNCTIONAL  SUBTEAMS  FACILITATES  THE  CONDUCT  OF  THE  PRR  ACTIVITIES. 
THIS  SLIDE  SHOWS  A  POSSIBLE  BREAKOUT  OF  THOSE  SUBTEAMS. 

AGAIN,  A  MAJOR  SYSTEM  ACQUISITION  MAY  HAVE  ALL  OF  THESE  ITEMS  AND  COULD  INCLUDE 
AS  MANY  AS  50  PEOPLE;  WHEREAS.  A  NON-MAJOR  SYSTEM  OR  A  VENDOR  TO  A  MAJOR  SYSTEM 
MAY  ONLY  HAVE  A  2-3  MAN  TEAM. 

APESO  ACTS  AS  AN  ADVISOR  TO  THE  TEAM  OR  AS  A  TEAM  PARTICIPANT  AND  DOD  PESO 
ACTS  AS  AN  INDEPENDENT  OBSERVER. 

THE  CORE  OF  THE  TEAM  SHOULD  BE  NON-PMO  PERSONNEL.  BUT  THEY  CAN  BE  SUPPLEMENTED 
WITH  KNOWLEDGEABLE  PERSONS  FROM  THE  PMO.  WE  ENCOURAGE  THE  USE  OF  THE  ON-SITE 
GOVERNMENT  PERSONNEL.  SOME  PM's  HAVE  HIRED  CONSULTANTS  TO  PARTICIPATE  IN  THE 
PRR. 
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CONDUCT  OF  THE  PRR 


THE  FINAL  PRR  REPORT 
e  FORMAL  REPORT  TO  PM 

•  IDENTIFIES  POTENTIAL  PROBLEMS 

•  PROVIDES  RISK  ASSESSMENTS:  COST/SCHED/ 
QUALITY/LOGISTICS 

•  RECOMMENDATIONS 

•  PM  PROVIDES  PRELIM  REPORT  TO  ASARC/DSARC/I PR 
MEMBERS  6  WEEKS  PRIOR  TO  PRE-REVIEW 


CONDUCT  OF  THE  PRR 

THE  FINAL  REPORT  IS  SUBMITTED  BY  THE  TEAM  CHIEF  TO  THE  PM.  IT  WILL  IDENTIFY 
THE  FINDINGS.  POTENTIAL  PROBLEMS.  ASSESSMENTS  OF  RISKS  IN  EACH  OF  THE  FUNCTIONAL 
AREAS  AND  RECOMMENDATIONS. 

THE  PM  SUBMITS  THE  FINAL  REPORT  TO  APESO  AND  THE  MILESTONE  III  DECISION  - 
MAKERS. 
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COMPLETED 

|  IN-PROCESS  \ 

1  NEXT  / 

|  12  MONTHS  / 

o  XM1 

1  •  AAH 

1  • DIVAD  / 

o  CH47 

•  e  HELLFIRE 

3  SINCGARRS  / 

o  MLRS 

•  VIPER 

I  e  PLRS/TIDS  / 

e  PATRIOT 

I 

|  o  MD918/GRC  / 

o  COPPERHEAD 

1  ^ 

|  o  30MM  AMMO  / 

o  GEMSS 

t  i 

i  0  STINGER/POST  / 

o  AN/APR  39 

| 

0  / 

e  IFV/CFV 

1  *  / 

ARMY  PRR  STATUS 

THIS  SLIDE  INDICATES  THE  HISTORY  OF  THE  ARMY-CONDUCTED  PRR's.  WE  HAVE  3  PRR's 
ON-GOING  NOW  AND  WE  ANTICIPATE  THAT  AT  LEAST  ANOTHER  6  WILL  BEGIN  DURING  THE  NEXT 
12  MONTHS.  IN  ORDER  TO  FIT  THE  PRR's  INTO  THE  APESO  SCHEDULES.  WE  HAVE  ASKED 
THE  PM's  IN  A  SEPARATE  LETTER  TO  PROVIDE  THEIR  PROJECTED  PRR  DATES. 
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ARMY  PRR  LESSONS  LEARNED 


•  EXPERIENCED  PERSONNEL 

•  INDEPENDENCE  OF  TEAM 

o  QUESTION -ASKING  VS.  ANALYSIS 

•  SIZE  OF  PRR  TEAM 

•  SINGLE  VISIT  VS.  INCREMENTAL 

•  COOPERATION  OF  CONTRACTORS 
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CONCLUSIONS 

ARMY  PRR  LESSONS  LEARNED 

AFTER  HAVING  CONDUCTED  PRR's  FOR  8  SYSTEMS  IN  THE  ARMY,  WE  SEE  THESE  FACTORS  AS 
BEING  HIGHLY  CONTRIBUTORY  TO  THEIR  SUCCESS: 

•  THE  USE  OF  PERSONNEL  WITH  EXPERIENCE  IN  PRIOR  PRR's,  PERHAPS  WITH  ANOTHER 
SYSTEM  AS  A  TRAINING  BASE,  WITH  EXPERTISE  IN  PRODUCTION  STARTUP  ACTIVITIES 
AND  WITH  POSITIVE,  FORTHRIGHT  ATTITUDES. 

•  THE  TEAM  SHOULD  BE  ABLE  TO  EXERCISE  INDEPENDENCE  OF  JUDGMENT.  IT  IS 
PARTICULARLY  IMPORTANT  FOR  THE  TEAM  CHIEF  AND  THE  SUB-TEAM  CHIEFS  TO  BE 
FROM  OUTSIDE  THE  PMO.  ON  THE  OTHER  HAND,  THE  PMO  PEOPLE  SHOULD  BE  PARTICI¬ 
PANTS  NOT  ONLY  FOR  THEIR  SPECIALIZED  KNOWLEDGE  BUT  TO  TAKE  IMMEDIATE  ACTION 
ON  PROBLEM  AREAS. 

•  THE  TEAM  MEMBERS  SHOULD  DEVELOP  DETAILED  CRITERIA  FOR  THEIR  OWN  FUNCTIONAL 
AREAS.  IN  A  NUMBER  OF  CASES,  THIS  PRIOR  PREPARATION  HAS  NOT  TAKEN  PLACE,  AND 
TEAM  MEMBERS  ARE  SIMPLY  GIVEN  A  LIST  OF  QUESTIONS  TO  ASK  THE  CONTRACTORS.  THE 
EMPHASIS  MUST  BE  PLACED  ON  ANALYSIS  OF  THE  PRODUCTION  READINESS  ACTIVITIES. 

•  THE  SIZE  OF  THE  PRR  TEAM  SHOULD  BE  GEARED  TO  THE  SIZE  AND  SCOPE  OF  THE  PROGRAM, 
IN  A  STUDY  CONDUCTED  BY  DOD  IT  WAS  FOUND  THAT  THE  RATIO  OF  CONTRACTOR  PERSONNEL 
TO  GOVERNMENT  PERSONNEL  INVOLVED  IN  THE  PRR  RANGED  FROM  1.5-2.  A  LARGE  TEAM 


CAN  BE  A  BURDEN  NOT  ONLY  TO  THE  CONTRACTORS,  BUT  TO  THE  EFFECTIVENESS 
OF  THE  REVIEW  AS  WELL.  ON  THE  OTHER  HAND,  THE  TEAM  MUST  BE  CAPABLE  OF 
HANDLING  ALL  ASPECTS  OF  PRODUCTION  READINESS  -  -  2  PERSONS  TO  A  FUNCTIONAL 
AREA  IS  RECOMMENDED  TO  PRECLUDE  OVERSIGHT  OF  PROBLEMS  AND  ENSURE 
INDEPENDENCE. 

•  IT  IS  EXTREMELY  DIFFICULT  TO  ASSESS  PRODUCTION  READINESS  IN  A  SINGLE  VISIT 
TO  A  CONTRACTOR  WITH  A  TEAM  OF  MANAGEABLE  SIZE.  INCREMENTAL  VISITS  THROUGH¬ 
OUT  ENGINEERING  DEVELOPMENT  DEVELOP  THE  CONFIDENCE  THAT  PRODUCTION  STARTUP 
ACTIVITIES  ARE  ON  SCHEDULE. 

•  FINALLY,  COOPERATION  OF  CONTRACTORS  CAN  ONLY  BE  ASSURED  IF  THE  REQUIREMENT 
FOR  PRR's  IS  WRITTEN  INTO  THEIR  CONTRACTS  AT  THE  BEGINNING  OF  ENGINEERING 
DEVELOPMENT. 
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CONDUCTING  INITIAL  PRODUCTION  READINESS  REVIEWS  (IPRR)  and 
PRODUCTION  READINESS  REVIEWS  (PRR) 


PRESENTED  BY:  LARRY  KENNEDY 

US  ARMY  AVIATION  RESEARCH  AND  DEVELOPMENT  COMMAND 

DIRECTORATE  FOR  PROCUREMENT  AND  PRODUCTION 
INDUSTRIAL  MANAGEMENT  AND  PRODUCTION  DIVISION  (DRDAV-PI) 
ST.  LOUIS,  MISSOURI  63120 
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DoD  Military  Weapons  Systems 
Preparation  for  and  Conducting  Initial  Production 

Readiness  Reviews  and  Production  Readiness  Reviews 


I  „  Preparation  for  the  IPRR/PRR 

A.  Selection  of  team  members 

1.  Limit  the  team  size  to  five  or  six  members  with  multi¬ 
discipline  experience,  if  possible. 

2.  Select  team  members  with  experience  in  areas  and  the 
type  systems/subsystems  to  be  reviewed. 

3.  Limit  the  team  to  one  member  (two  for  larger  teams)  from 
the  PMO  of  the  system  being  reviewed. 

4.  Plan  for  an  advance  team,  consisting  of  the  IPRR/PRR 
manager  and  another  member  of  the  team,  to  have  a 
preliminary  PRR  meeting  with  PMO,  the  prime  contractor, 
and  the  subcontractors  initially  selected  to  be  visited. 

B.  The  preliminary  IPRR/PRR  meeting 

1.  At  this  meeting,  present  to  the  PMO,  prime  contractor,  and 
selected  subcontractors  the  objectives,  planning,  method¬ 
ology,  and  procedures  to  be  used  in  conducting  the  Review 
at  each  contractor's  site.  Advise  the  contractors  of  the 
general  type  of  information  and  validation  being  sought, 
the  caliber  of  people  that  are  on  the  team  and  that  you 
expect  to  interface  with  the  review  team. 

2.  Develop,  from  this  meeting,  the  list  of  p art s / s ys terns  to 
be  supplied  by  the  prime  and  each  subcontractor  you  plan 
to  visit. 

3.  From  the  list  and  in  the  meeting  determine  the  system 
complexity  and  potential  depth  of  review  necessary  for 
each  subcontractor.  This  list  should  also  highlight  the 
need  for  any  special  talent  expertise  that  may  be  required 
on  the  review  team  to  assess  special  technologies  that 
may  be  encountered. 

4.  Establish  and  review,  if  any,  the  past  history  and 
experience  of  the  PMO  and  prime  contractor  with  each 
subcontractor  for  production  of  the  same,  similar  or 
related  parts  or  systems. 
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C.  Selection  of  topics  and  areas  to  be  reviewed  in  the  IPRR/PRR 

1.  Develop  for  review  at  the  prime  contractor's  and  each 
subcontractor's  facilities  questions  tailored  to  the 
specific  type  of  operation/  contract ,  and  components 

or  systems  being  supplied  by  the  individual  contractor. 

DODI  5000.38  can  be  used  as  a  topic  and  general  guide, 
but  more  pertinent  questions  should  be  developed  for 
each  type  of  plant. 

2.  Check  with  the  in-plant  government  representative  or 
office  or  responsible  government  contract  administrative 
office  for  the  past  history  and  current  experience  of 
the  contractor  on  government  contracts  or  related 
commercial  items. 

3.  Determine  the  programs/projects,  and  number  of  other 
related  military  and  commercial  contracts  each  contractor 
has  scheduled  for  production  during  the  same  time  frame 
the  program  under  review  is  to  be  in  planning  and 
production . 

4.  Determine  in  the  questions  prepared  for  each  contractor 
to  be  visited,  the  impact  of  concurrent  production 
contracts,  their  relative  priority  ratings,  schedules, 
and  production  equipment  utilization  requirements. 

D.  Establishing  the  agenda 

1.  Visits  to  contractors'  plants  should  be  planned  by 
geographic  areas  -  to  maximize  the  time  on-site  and  minimize 
travel  time  and  cost. 

2.  Review  time  scheduled  at  each  site  should  be  in  relation 
to  the  type  of  contract,  production,  and  operation 
involved,  e.g.  a  forging  or  casting  facility  producing 
parts  to  the  prime  contractor's  specifications  for  rough 
finish  and  quality  requires  little  time. 

3.  Submit  the  proposed  questions,  agenda  and  schedule  to  the 
PMO,  prime  contractor  and  other  sys terns -supplie rs  ( GFE 

PMOs  and  contractors)  for  comment  and  schedule  coordination. 
The  prime  contractor  should  be  given  the  subcontractor's 
questions  and  information  for  distribution  to  them  and 
coordination  of  visits.  Return  comments  on  the  submitted 
agenda  and  questions  should  be  requested,  if  time  permits. 

4.  Coordinating  with  the  PMO  and  prime  contractor  allows 
adequate  time  in  planning  for  contractors'  comments  and 
replies  to  be  received  and  answered. 
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5. 


Review  and  revise  IPRR/PRR  questions,  agenda,  and 
schedules  to  reflect  updated  information  gained  during 
planning  exchanges  and  programs  history  reviews.  (This 
includes  review  of  DoD  files  of  the  past  performance 
history  of  the  contractor  on  previous  programs.) 

IX.  Conducting  the  IPRR/PRR 

A.  Organizing  the  team  effort 

1.  Have  an  initial  assembly  and  briefing  of  the  IPRR/PRR 
team  at  the  program  office  describing  the  system  and 
planned  program. 

2.  Question  the  PMO  staff  as  to  problems,  concerns,  past 
problems  resolved,  schedules,  long  lead  items,  and 
program  alternatives. 

3.  From  the  program  office  staff  elements  obtain  assess¬ 
ments  of  the  critical  areas  and  risk  factors. 

B.  On-site  reviews 

1.  The  initial  visit  should  be  to  the  prime  contractor's 
facilities.  A  briefing  as  to  their  status,  their 
assessment  of  each  sub-contractor  and  the  overall  status 
and  interplay  all  elements  of  the  program  should  be 
presented  here.  The  final  visit  of  the  IPRR/PRR  team 
also  should  be  made  to  the  prime  contractor's  facilities. 
This  time,  to  determine  if  any  changes  are  occurring 

and  to  advise  the  prime  of  the  Review  findings  at  the 
subcontractors  and  to  inform  them  of  any  areas  of 
c once rn  . 

2.  At  the  prime's  facilities,  the  team  should  receive  an 
initial  briefing  from  the  DCASR  or  in-plant  government 
representative  office  as  to  their  assessment  of  the 
contractor's  development  progress  and  production 
preparedness.  The  final  visit  to  the  prime  contractor 
should  provide  for  an  update  by  the  government  represen¬ 
tative  as  to  any  changes,  corrections  or  schedule 
revisions  that  may  have  resulted  from  test  failures  or 
other  problems  that  may  have  developed. 

3.  Visits  to  the  prime  and  subcontractors  should,  following 
instructions,  be  kicked-off  with  questioning  of  the 
contractor  as  to  any  problems  encountered  with  the 
submitted  questions.  This,  followed  by  a  brief  presen¬ 
tation  of  the  company's  organization,  operations,  and 
part  in  the  program  being  reviewed,  eliminates  a  later 
discovery  that  they  are  not  really  prepared  to  present 
the  material  and  information  needed. 
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4.  A  plant  tour  of  those  facilities  that  are  to  be  utilized 
or  will  support  production  related  to  the  reviewed  system 
is  desirable.  The  tours  should  be  tailored  to  cover  the 
specific  program  production  equipment  and  related  opera¬ 
tions  . 

5.  The  team  review  at  each  site  should  endeavor  to  cover  all 
elements  of  the  topics  of  the  IPRR/PRR  related  to  that 
contractor.  The  questions  submitted  to  the  contractors 
in  advance  of  the  visits  are  to  serve  as  openers  only 
and  should  lead  the  reviewing  team  members  into  other  and 
deeper  questions  as  topics  and  areas  of  concern  are  re¬ 
vealed  . 

6.  Encouragement  to  fellow  team  members  to  look  into  areas 
of  question/concern  which  may  give  cause  or  evidence  of 
a  problem  in  other  areas  is  a  necessary  element  of  team 
crossfeed  during  the  review. 

C.  Completing  the  job 

1.  A  wrap-up  visit  to  the  prime  contractor  and  to  the  PMO 
should  be  the  final  visits  in  the  reviews.  Even  these 
visits  should  be  a  two  way  exchange.  The  PMO  should  be 
presented  the  preliminary  report  findings  of  the  review 
team  for  use  in  developing  action  items  to  be  pursued 
prior  to  D S ARC .  The  PMO  should  also  inform  the  review 
team  of  any  problems  or  areas  of  concern  that  they  are 
aware  of  and  any  actions  taken  to  resolve  them. 

2.  Similarly,  the  final  review  of  the  prime  contractor’s 
plant  should  inform  them  of  the  problems/concerns  found 
by  the  Review  team  and  give  them  opportunity  to  initiate 
corrective  actions  prior  to  final  formulation  of  the 
IP'RR/PRR  report.  The  contractor  should  be  questioned  at 
this  time  as  to  any  outstanding  actions  scheduled  to  be 
completed  prior  to  preparation  and  submittal  of  the  final 
report.  This  meeting  should  also  produce  a  checksheet  of 
the  data  items  the  contractor  still  needs  to  provide  the 
Review  Team  and  any  open  items  yet  to  be  reviewed  or 
reports  are  due  on. 

III.  The  IPRR/PRR  Report 

A.  Format  -  The  remaining  part  of  the  review  effort  is  pre¬ 
paration  of  the  report  which  will  vary  by  service  and  command 

preference  for  which  usually  the  format  is  prescribed. 

B.  Report  Essentials 

1.  Essential  to  a  good  report  is  documenting  the  program  areas 
of  weakness  and  concern  found  in  the  Review. 
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to  correct 


Actions  planned  by  the  PMO  and  the  contractors 
these  problems  should  also  be  reported. 

Risk  areas  and  the  contributing  factors  should  be  headlined 
with  realistic  assessments  of  their  impacts  on  the  pro¬ 
gram's  production. 

Any  incomplete  reports,  tests,  or  data  denied  or  deter¬ 
mined  inaccessible  should  be  noted.  \ 

Above  all  the  report  should  represent  the  unified  findings 
and  opinions  of  the  IPRR/PRR  team  members. 
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CH-A7  MODERNIZATION 
PRODUCTION  READINESS 
REVIEW  PLAN 


Presented  by:  LARRY  KENNEDY 

US  Army  Aviation  Research  and 
Development  Command 
Directorate  for  Procurement  and 
Production 

Industrial  Management/Production 
Division  (DRDAV-PI) 

St.  Louis,  MO  63120 


INITIAL  PRODUCTION  READINESS  REVIEW 

(IPPP) 

THE  FOLLOWING  REGULATIONS  ESTABLISH  THE  CRITERIA  FOR  IPRRS  AND  PRRS: 

A,  ARMY  PRODUCTION  READINESS  REVIEW  REGULATION  70  -  67,  1  DEC  79 

B,  DOD  5000,1,  MAJOR  SYSTEM  ACQUISITION,  18  JAN  77 

C,  DOD  5000,2,  MAJOR  SYSTEM  ACQUISITION  PROCESS,  18  JAN  77 
1,  DOD  5000.34,  DEFENSE  PRODUCTION  MANAGEMENT,  31  OCT  77 
E,  D0DI  5000,38,  PRODUCTION  READINESS  REVIEWS,  2A  JAN  79 
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EXAMPLES  OF  PRODUCTION  READINESS  CRITERIA 


•  LOW  RISK  PRODUCT  DESIGN  IN  TERMS  OF  PRODUCIBILITY 

•  ECP  ACTIVITY  HAS  STABLIZED  AT  A  LOW  LEVEL 

•  TECHNICAL  DATA  PACKAGE  IS  COMPLETE/PROPRIETARY  ITEMS  ARE  KNOWN 

•  pr°ducti°n  sched,jles  with  end  item 

•  CRITICAL/SCARCE  MATERIALS  USED  ONLY  WHERE  DICTATED 

•  COMPREHENSIVE  MANUFACTURING  PLAN  DEVELOPED 

•  VALUE  ENGINEERING  AND  QUALITY  PROGRAMS  DEVELOPED 


PRODUCTION  READINESS  REVIEW  ORGANIZATION 


CH-47  MOP  PROGRAM 
IPRR/PRR  REVIFW  AREAS 


TEAM  CHIEF 


1 


PRODUCTION 

TECHNICAL 

LOGISTICS 

MANAGEMENT/ 

CONTRACTS 

DESIGN  PRODUCIBITY 
PRODUCTION  PLANNING 
FACILITIES/TOOLING 
PROCESS  PLANNING 
MANPOWER 
DESIGN-TO-COST 
MATERIALS/MAKE  BUY 
GFM/GFE 


DESIGN  ENGINEERING 

CONFIGURATION 

TESTING 

QUALITY  ASSURANCE 
TECHNICAL  DATA  PKG 
VALUE  ENGINEERING 


HjNJNG  MANUALS  & 

TRANSPORTATION 

SPARES 

ILS  PROGRAM 

OSS  COST  ESTIMATES 


ROTARY  DESIGNS, 

LABOR  RELATIONS/EEO 

OSHA/FN'M  RONMENTAL/ 
ENERGY 

CONFIGURATION  MGT 

mmsm 
mem®-  fa 

MI  SC 

SUBCONTRACTORS  B 
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DECISIONS  AFFECTED  BY  PRR 


PRIMARILY: 


•  ASARC  III 

•  DSARC  III 

I  AND,  ULTIMATELY,  THE  SECDEF'S  DECISION 


CH-97  INITIAL  PRODUCTION  READINESS 
AND  FORMAL  PRODUCTION  READINESS  REVIEWS 

•  INITIAL  MEETING  WITH  BOEING-VERTOL 

•  PURPOSE 

•  SCOPE 

•  ESTABLISH  SUBCONTRACTORS  SCHEDULE 

•  LIST  OF  BOEING  SUBCONTRACTORS 

HONEYWELL  CORPORATION  (IPRR) . 7  -  8  AUG  ?9 

RONSON  HVDRA'JLIC  UNITS  CORPORATION  (IPRR) . 9  -  10  AUG  70 

RONSON  HYDRAULIC  UNITS  CORPORATION  (PRR) . 10  -  11 


76 


AND  fORMAI  PRODUCTION  READ! NFSS  REVIEWS  (CONTINUES! 


WYMAN  GORDON  CORPORATION  (PRP.) . 1°  -  15  AUG  79 

FEEN MANUFACTURING  COMPANY  (PRR) . 21  -  22  AUG  79 

LITTON  PRECISION  GEAR  (IPRR) . 28  -  29  AUG  79 

LITTON  PRECISION  GEAR  (PRR) . 26  MAR  80 

AVCO  LYCOMING  (IPRR)  GFM . -  18  OCT  79 

AVCO  LYCOMING  (PRR  INTERIM)  GFM . 15  -  18  APR  80 

AVCO  LYCOMING  (FINAL  PRR)  GFM . 18  JUN  80 


CH-97  INITIAL  PRODUCTION  READINESS 
AND  FORMAI  PRODUCTION  READINESS  REVIEWS  (CONTINUED) 


nOEING  VERTOL  (IPRR) . 

BOEING  VERTOL  (PRR) . 

AIRCRAFT  POROUS  MEDIA  INC.  (PRR) 

NEW  YORK  AIR  BRAKE  (PRR) . 

LADISH  PACIFIC  COMPANY  (PRR).... 
SUNDSTRAND  CORPORATION  (PRR).... 
TURROMACH  (PRR)  GFM . 


1  -  5  OCT  79 
29  -  2  MAY  80 
.9  -  6  FEB  80 
13-15  FEB  80 
,12-13  MAR  80 
,29-26  MAR  80 
,?9  -  ?  JUL  80 
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COPPERHEAD 

PRODUCTION  READINESS  REVIEW 

LESSONS  LEARNED 


PRESENTED  BY 

JIM  PRITCHARD 


TITLE  CHART 


Good  Afternoon: 

It  is  indeed  a  pleasure  to  be  here  today  to  talk  about  my  experiences 
on  the  COPPERHEAD  Production  Readiness  Review.  I  might  also  add  that  I 

AM  CONVINCED  AS  TO  THE  VALUE  OF  THE  PRR  AND  THINK  THAT  THIS  CONFERENCE, 
WHICH  IS  CONCERNED  WITH  IMPROVING  PRR'S,  IS  AN  EXCELLENT  EFFORT. 
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OUTLINE 


•  BACKGROUND 

•  PRR  PHASES 

•  PLANNING 

•  FOLLOW-UP 

•  "LOOKING  BACK" 

•  SUMMARY 


OUTLINE  CHART 


Shown  here  is  an  outline  of  my  presentation,  First  I  would  like  to 

GIVE  A  LITTLE  BACKGROUND  ON  THE  ITEM,  FACILITIES,  AND  THE  PRR.  THEN  I 
WILL  DISCUSS  THE  THREE  PHASES  OF  THE  PRR,  A  DISCUSSION  ON  PLANNING, 
IMPORTANCE  AND  VALUE  OF  FOLLOW-UP  ,  A  LOOK  BACK  AT  OUR  PRR,  AND  SOME 
SUMMARY  REMARKS. 
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PRR  SCHEDULE 


INITIAL  PRR  (DARCOM) 

PRR  (AT  CONTRACTOR) 

PRR  UPDATE  (AT  CONTRACTOR) 
PRR  UPDATE  (AT  ARRADCOM) 
PRR  UPDATE  (AT  DARCOM) 


DECEMBER  18,  1978 
MARCH  7-9,  1979 
MAY  2,  1979 
MAY  29,  1979 
JUNE  30,  1979 


PRR  SCHEDULE  CHART 


Shown  on  this  chart  is  our  PRR  schedule,  our  main  PRR  was  held  a 

YEAR  AND  A  HALF  AGO  WITH  THREE  (3)  SUBSEQUENT  UPDATES, 
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THE  ITEM 


•  155MM  PROJECTILE 

•  LASER  GUIDED 

•  HARDENED  ELECTRONICS 

•  SHAPED  CHARGE  WARHEAD 

•  CONTRACTOR:  MARTIN  MARIETTA 

AEROSPACE 


THE  ITEM  CHART 

The  COPPERHEAD  is  a  snake  but  what  we  are  concerned  about  in  this 

DISCUSSION  IS  A  155MM  PROJECTILE  WHICH  IS  SHOT  FROM  A  HOWITZER  AND  IS 
LASER  GUIDED  TO  THE  TARGET  BY  MEANS  OF  A  DESIGNATOR,  In  ORDER  TO  PER¬ 
FORM  ITS  FUNCTION/  THE  ELECTRONICS  PACKAGE  ASSOCIATED  WITH  GUIDANCE  AND 
CONTROL  HAD  TO  BE  HARDENED  IN  ORDER  TO  SURVIVE  THE  10/000  G  GUN  LAUNCH 
ENVIRONMENT,  The  ROUND  CONTAINS  A  SHAPED  CHARGE  WARHEAD  CONTAINING 

Composition  B  explosive.  Prime  contractor  is  Martin  Marietta  Aerospace 
Corporation/  Orlando,  Florida  who  was  selected  as  the  result  of  a  shoot- 

off  DURING  ADVANCE  DEVELOPMENT.  In  ADDITION  TO  MARTIN/  THERE  ARE  FIVE 
MAJOR  SUBCONTRACTORS,  LOADING  IS  DONE  BY  IoVfA  IAAP, 
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THE  PROCESS 


PRIME  CONTRACTOR 

•  ELECTRONICS  ASSEMBLY  &  TEST 

•  METAL  WORKING 

•  OPTICS 

•  GYRO 

SUBCONTRACTORS 

•  FUZING 

•  LSI'S 

•  CONTROL  SYSTEM 

•  ROLL  RATE  SENSOR 

•  BATTERY 

2ND  TIER  VENDORS 

•  ELECTRONIC 

•  MECHANICAL 

GOVERNMENT  LOADING 


THE  PROCESS  CHA^T 

Continuing  to  give  more  background  on  the  PP.R,  this  chart  indicates 

SOME  OF  THE  CHARACTERISTICS  OF  THE  PROCESS.  THE  PRIME  CONTRACTOR'S 
EFFORT  IS  BASICALLY  ELECTRONICS  SUB* ASSEMBLY,  ASSEMBLY,  TEST  AND  FAB¬ 
RICATION.  In  addition  he  machines  the  projectile  housings.  In  addi¬ 
tion  HE  HAS  SET  UP  A  NUMBER  OF  SPECIALIZED  MODULES  FABRICATING  OPTICS, 
GYROS,  AND  COILS.  PRIME  SUBCONTRACTORS  ARE  INVOLVED  WITH  FUZE,  LARGE 
SCALE  INTEGRATED  CIRCUITS,  FLIGHT  CONTROL  SYSTEMS,  ROLL  RATE  SENSORS, 
AND  A  BATTERY.  BEYOND  THE  SUBCONTRACTORS  ARE  THE  2nd  TIER  VENDORS 
SUPPLYING  THE  PRIMARLY  ELECTRONIC  AND  MECHANICAL  COMPONENTS.  IOWA 

Army  Ammunition  Plant  loads  our  warhead. 
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PRR  PHASES 


•  PLANNING 

•  THE  PRR 

•  FOLLOW-UP 


PRR  PHASES  CHART 


NOW  THAT  1  HAVE  GIVEN  YOU  A  LITTLE  BACKGROUND  ON  OUR  ITEM/  AND 
PRR/  LET  ME  GIVE  YOU  SOME  OF  MY  OBSERVATIONS. 

I  CONSIDER  THE  PRR  TO  BE  IN  THREE  PHASES  AS  SHOWN  ON  THIS  CHART, 

The  punning  phase  which  proceeds  the  actual  PRr,  the  PRR  itself,  and 

PRR  FOLLOW-UP,  I  WILL  DEVOTE  MOST  OF  MY  REMARKS  TO  THE  FIRST  AND  LAST 
PHASES  SINCE  [  CONSIDER  THEM  TO  BE  EXTREMELY  IMPORTANT. 
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PLANNING 


•  FORMATION  OF  PLANNING  GROUP 

•  COORDINATION  WITH  CONTRACTOR 

•  DETERMINATION  OF  NEEDS  U0 

•  DEVELOPMENT  OF  ORGANIZATION 

•  IDENTIFICATION  OF  PERSONNEL 

•  SELECTION  OF  PERSONNEL  V0 

•  TRAINING  OF  PERSONNEL  V0 


PLANNING  CHART 


On  this  chart  I've  shown  some  of  the  major  steps  in  the  planning 

STAGE  AND  WOULD  LIKE  TO  DISCUSS  THE  CHECKED  ITEMS,  It  IS  DIFFICULT 
TO  PLAN  PROPERLY  IF  A  THOROUGH  DETERMINATION  OF  NEEDS  IS  NOT  MADE.  In 
OUR  CASE/  A  BROAD  SPECTRUM  OF  DIVERSE  TECHNOLOGIES  NECESSITATED  A  WIDE 
RANGE  OF  INDIVIDUAL  SKILLS  IN  ORDER  TO  PERFORM  THE  PRR, 

Selection  of  the  right  people  is  the  most  important  planning  func¬ 
tion,  Our  experience  has  shown  that  you  should  exhibit  most  demanding 

PERSISTANCE  IN  YOUR  SEARCH  AND  SELECTION  OF  THE  RIGHT  PEOPLE.  AFTER 
FINDING  THEM/  THEY  SHOULD  BE  TRAINED/  IF  NOT  FAMILIAR  WITH/  THE  PRR 
PROCESS, 
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FOLLOW-UP 

•  PRR  IS  A  CONTINUING  FUNCTION 

•  ACTIONS  ARE  INITIATED:  NEED  FOLLOW-UP 

•  UTILIZE  THE  SKILLS  AVAILABLE 

•  "KEEP  THE  PRESSURE  ON" 


FOLLOW-UP  CHART 

Leaping  over  the  PRR  from  the  planning  stage  into  the  follow-up, 

THIS  CHART  DISPLAYS  SOME  OF  THE  KEY  POINTS  OF  THIS  PHASE.  PRR  IS 
AN  INVESTMENT  -  IT  IS  A  CONTINUING  FUNCTION.  ACTIONS  ARE  IDENTIFIED, 
INITIATED,  BUT  NEED  FOLLOW-UP.  SOME  OF  THE  SKILLS  WHICH  HAVE  BEEN 

assembled  can  be  used  also  in  a  follow-up  mode.  The  key  is  keep  the 

PRESSURE  ON  -  GAIN  THE  BENEFITS  OF  THE  PRR. 
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"LOOKING  BACK" 


•  RAN  OUT  OF  TIME 

•  PRR  COULD  BE  DONE  IN  2  1/2  DAYS 

•  MOST  ASSUMPTIONS/CONCERNS  WERE  VALID 

•  EXPERIENCE  CAN  IMPROVE 


"LOOKING  BACK"  CHART 

Looking  back  at  our  PRR  it  seemed  as  if  we  just  ran  out  of  time 

IN  THE  PLANNING  STAGE,  As  WE  WERE  GETTING  THE  ENTIRE  PRR  TEAM  UP  TO 
SPEED  THE  BIG  DAY  WAS  UPON  US.  ADDITIONAL  TIME  WOULD  HAVE  BEEN 
VALUABLE  -  MORAL  HERE  -  BUDGET  AND  PLAN  TIME. 

I  WAS  VERY  SKEPTICAL  THAT  WE  COULD  DO  WHAT  WE  HAD  TO  DO  IN  ONLY 
2h  DAYS.  I  WAS  QUITE  PLEASANTLY  SUPRISED  THAT  WE  DID  A  DECENT  JOB  IN 
THAT  TIME  FRAME.  MORAL  HERE  IS  TO  DO  WHAT  YOU  CAN  AHEAD  OF  TIME  - 
ACTUAL  PRR  TIME  IS  GOLDEN. 

A  MOST  REASSURING  FACT  WAS  THAT  MOST  ASSUMPTIONS  MADE  AND  CONCERNS 
DID  PROVE  VALID.  A  GOOD  ANALYSIS  WAS  MADE. 

Bottom  line  is  that  based  on  our  experience  we  could  improve  over 

WHAT  WE  FELT  WAS  A  GOOD  PRR. 
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SUMMARY 


•  PLAN  CAREFULLY 

•  GET  THE  RIGHT  PEOPLE 

•  TRAINING 

•  FOLLOW-UP 


SUMMARY  CHART 


In  summary,  our  experience  has  shown  you  should  plan  very  carefully 

AND  VERY  EARLY  -  IT  ALWAYS  SEEMS  THERE  IS  NOT  ENOUGH  TIME  LEFT  BEFORE 

the  PRR,  People  are  the  key  to  success  for  the  PRR  -  Determine  the 

NEEDS,  SEARCH  OUT  THE  RIGHT  PEOPLE,  AND  "LAND"  THEM.  TRAIN  PEOPLE  FOR 
THE  PRR,  YOUR  PRR,  YOUR  ITEM,  IF  THEY  ARE  NOT  KNOWLEDGEABLE  IN  THESE 

areas.  Save  the  "golden  time"  of  the  actual  PRR  for  what  is  essential  - 

CERTAINLY  NOT  TRAINING  THAT  COULD  BE  DONE  AHEAD  OF  TIME. 
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AAH 

PRODUCTION  READINESS  REVIEW  PLAN 


2472 


PRODUCTION  READINESS  REVIEW 

AGENDA 

ANTHONY  PIAZZA 

m  INTRODUCTION  AND  ORIENTATION 

•  IPRR/PRR  OVERVIEW 

•  IPRR/PRR  DEFINITIONS 

•  IPRR/PRR  SCHEDULES 

#  AAH  PRR  APPROACH 
m  TEAM  ORGANIZATION 
f»  AREAS  OF  REVIEW 

2474 
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OPTEB 


INITIAL  PRODUCTION  READINESS  REVIEW 

(IPRRj  AGENDA 


•  INDIVIDUAL  FUNCTIONAL  TEAM  CAPTAIN  MEETINGS 

•  DISCUSS  CRITERIA 

•  DISCUSS  REQUIRED  EVIDENCES 

•  REVIEW  CURRENT  STATUS/AVAILABLE 
DATA/PROBLEM  AREAS 

•  EXIT  INTERVIEW 

•  AREAS  OF  CONCERN 

•  GUIDANCE/REQUIRED  DATA  FOR  PRR 


AAH  PRODUCTION  TRANSITION 


1979 

1980 

1981 

1982 

1983 

ASARC  DSA 

▼  T 

RC 

PEP 


MAY 


NOV 


IPRR’S 


JUN 


NOV 


DTUPC  REV 


JUL 


IPRR’S 


"I - 

JAN  APR  SEP 

|  PRR’S 


JUL _ JAN  APR  SEP 

DTUPC  REV  I  D]TUPC  REV 


LLTI  AWD 


PROD  AWD 


▼ u 

LONG  LEAD 

PROCUREMENT 

  1  1  - 

LONG  LEAD 

TOOL  FABRICATION 

L 1 

DETAIL  FABRICATION 

SUBASSEMBLY 
•'I 


J 


LINE  FLOW 


J 


MAJOR  ASSY  ) 


INSPECT  l 


ramp/ 


1ST  A/C  DELIVERY 
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2308 


PRR  MILESTONES 


IPRR  PERFORMANCE.. . 

. APR  -  SEP  79 

IPRR  COMPLETE . 

. .1  NOV  79 

INPRR  PERFORMANCE . 

. AUG  80 

INPRR  COMPLETED . 

. JAN  81 

PRR  PERFORMANCE . 

. APR  -  JUL  81 

PRR  COMPLETE . 

. .1  SEP  81 

ASARC  III . 

. 15  NOV  81 

DSARC  III . - . 

. 5  DEC  81 

PRODUCTION  AWARD . 

. 10  DEC  81 
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TRANSITIONING  TO  PR0D0CTI0N 


•  REVIEW  OF  BLACK  HAWK  PRODUCTION  EXPERIENCE 

•  DETERMINE  LESSONS  LEARNED 

•  SIGNIFICANT  PEP  EFFORT  CURRENTLY  IN  BEING 

•  MASTER  PROGRAM  AND  PRODUCTION  SCHEOULES  COMPLETEO 

•  COMPREHENSIVE  PRODUCTION  PLAN  UNOER  DEVELOPMENT 

•  IN-HOUSE  PRODUCTION  ORAWINGS  APPROXIMATELY  60%  COMPLETE 

•  TOOL  DESIGN  APPROXIMATELY  32%  COMPLETE 

•  CONTRACTOR  FACILITIZATION  PLANS  TO  MEET  PRODUCTION  OELIVERY 
SCHEOULE 

•  FACILITY  PLANS  TOTALING  APPROXIMATELY  $50M  BEING 
IMPLEMENTED  ANO  ACHIEVEMENTS  RIGOROUSLY  MONITOREO 

2475 
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TRANSITIONING  TO  PRODUCTION  (CONT’D) 


•  EARLY  SELECTION  AND  PLACEMENT  DF  KEY  MEMBERS  OF  CONTRACTOR'S 
PRODUCTION  TEAM 

•  KEY  MEMBERS  SELECTED  OR  TO  BE  SELECTED  IN  THE  NEAR  FUTURE 

•  PRDDUCTIDN  READINESS  REVIEWS  (PRRs) 

•  INITIAL  PRODUCTION  READINESS  REVIEWS  CONDUCTED  IN  CY  79 

•  SELECTED  INTERMEDIATE  PRRs  CURRENTLY  BEING  CONDUCTED 

•  FINAL  PRRs  SCHEDULED  APR  THRU  SEP  81 

•  DA  AND  DOD  PESO  PARTICIPATION 

•  PRODUCTION  SCHEDULE  COMPATIBLE  WITH  END  ITEM  DELIVERY 
REQUIREMENTS 

•  LLTI/IPF  REQUIREMENTS  IDENTIFIED  AND  BUDGETED  (34  MONTH  PLT) 
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DECISIONS  AFFECTED  BY  PRR 

PRIMARILY: 

•  ASARC  III 

•  DSARC  III 

•  AND,  ULTIMATELY,  THE  SECDEF'S  DECISION 
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PRODUCTION  LESSONS  LEARNED 


WHAT  WE  LEARNED  FROM  HACK  HAWK  WHAT  AAH  HAS  DONE/WILL  DO 

DIFFERENT 

•  PRODUCTION  LEADTIME  TOO  SHORT  •  PRODUCTION  LEADTIME  REVISED 

(20  MONTHS)  FROM  25  TO  34  MONTHS  IN  EARLY  79 

.  LATE  IDENTIFICATION  OF  LLTI  •  EARLY  IDENTIFICATION  AND  BUDGET 

FOR  LLTI 

•  LATE  INITIATION  OF  PEP  •  PEP  INITIATED  CONCURRENT  WITH 

START  OF  PHASE  2 

•  MINIMUM  PRODUCTION  READINESS  •  PROGRESSIVE  PRODUCTION 
VERIFICATION  ACCOMPLISHED  PRIOR  READINESS  REVIEWS  ON  PRIMES  & 

TO  PRODUCTION  AWARD  ALL  MAJOR  SUBCONTRACTORS 

.  MANUFACTURING  PLANNING/  •  MANUFACTURING  &  FACILITIES  PLANS 

FACILITATION  ACCOMPLISHED  IDENTIFIED  EARLY  AND  RIGOROUSLY 

CONCURRENT  W/INITIAL  PRODUCTION  MONITORED  (QUARTERLY 

PRODUCTION  AND  FACILITIES  PLAN) 

•  LATE  COMPLETION  OF  DESIGN  AND  •  CLOSELY  MONITOR  DESIGN  AND 

QUALIFICATION  QUALIFICATION  EFFORT 
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INTERMEDIATE  PRODUCTION 
READINESS  REVIEW 

(IN-PRR)  AGENDA 


IT  IS  A  FORMAL  EXAMINATION  OF  PROGRAM 
TO  DETERMINE: 

•  IF  THE  LONG  LEAD  ITEMS  DESIGN  IS 
READY  FOR  PRODUCTION 

•  LEVEL  OF  PRODUCTION  ENGINEER 
PROBLEMS  (BENCH  MARK) 

•  STATUS  OF  PLANNING 
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PRODUCIBILITY  SUMMARY 


•  BLACKHAWK  PRODUCTION  PROBLEMS  REVIEWED  FOR 
“LESSONS  LEARNED” 


•  PEP  EFFORT  IN  BEING  AT  HUGHES,  MARTIN  &  MAJOR 
SUBCONTRACTORS 


•  PRODUCTION  READINESS/DTUPC  REVIEWS  IN 
PROGRESS 


•  LLTIs  IDENTIFIED  AND  BUDGETED 
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PRR  SCHEDULE 


CONTRACTOR 

ASD 

BERTEA 

SPERRY 

GARRETT  AI RESEARCH 

TELEDYNE  SYSTEMS 

ROCKWELL  INTERNATIONAL 

LITTON  GUIDANCE 

BENDIX 

a/c  GEAR 

LITTON 


SUBSYSTEM  COMPONENT 
ROTOR  BLADES 
HYDRAULICS 

ASE  SYMBOLOGY  GENERATOR  MULIPLEX 
APU 

FIRE  CONTROL  COMPUTER 

HELLFIRE 

HARS 

DRIVE  SHAFTS  COUPLING 
INTERMEDIATE  AND  t/r  GEARBOXES 
TRANSMISSION  NOST  GEARBOXES 
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PRR  SCHEDULE 


CONTRACTOR 

TELEDYNE  RYAN 
ILS 

MARTIN  MARIETTA 
MENASCO 

HONEYWELL 

RCA 

HUGHES 

GARRET  AIRESEARCH  (CA> 
REPORT  COMPLETE 


subsystem/component 

FUSELAGE 
LASER 
TADS/PNVS 
LANDING  GEAR 

THADSS 
AIE 
AAH 
I  PAS 


AAH  WEAPONS  SYSTEM 
PRQPOSFD  PRR  APPROACH 

•  MULTI-TIERED  TEAM  APPROACH 

•  HUGHES 

•  TELEDYNE  RYAN 

•  OTHER  MAJOR  SUBCONTRACTORS 

•  CONSOLIDATED  PRR  REPORT  FOR  ASARC  1 1 1/DSARC  III  ADDRESSING 
AAH  AS  AN  INTEGRAL  SYSTEM 
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IPRR/PRR  TEAM  ORGANIZATION 


IPRR/PRR  REVIEW  AREAS 
(EXAMPLE  -  LEVEL  “A") 


PRODUCTION 


MANAGEMENT/- 

CONTRACTS 


DESIGN  PRODUCIBILITY 
PRODUCTION  PLANNING 
FACILITIES/TOOLING 
PROCESS  PLANNING 
MANPOWER 
DESIGN-TO-COST 
MATERIALS /MAKE-BUY 
CRITICAL/ SCARCE  MAT' LS 
GFM/GFE 


DESIGN  ENGINEERING 

CONFIGURATION 

TESTING 

QUALITY  ASSURANCE 
RATIONALIZATION, 
STANDARDIZATION, 
INTEROPERABILITY 
TECHNICAL  DATA  PACKAGE 
VALUE  ENGINEERING 


PSE 

TRAINING  MANUALS  & 
DEVICES 

TRANSPORTATION 

SPARES 

ILS  PROGRAM 

O&S  COST  ESTIMATES 

TRANSITION  PLANS 


PROPRIETARY  DESIGNS, 

MAT ' LS ,  PROCESSES 
LABOR  RELAT ION S / EEO 
OSHA/ ENVIRONMENTAL/ 
ENERGY 

CONFIGURATION  MANAGEMENT 
MANUFACTURING  ORG/ 
POLICIES/PROCEDURES 
CONTRACTOR'S  PROCURE¬ 
MENT  SYSTEM 
MISC 

SUBCONTRACTORS 
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EXAMPLE  OF  PRODUCTION  READINESS  CRITERIA 

LOW  RISK  PRODUCT  DESIGN  IN  TERMS  OF  PRODUCIBILITY 
ECP  ACTIVITY  HAS  STABILIZED  AT  A  LOW  LEVEL 
TECHNICAL  DATA  PACKAGE  IS  COMPLETE 
LONG  LEAD  ITEMS  ORDERED 

CRITICAL/SCARCE  MATERIALS  USED  ONLY  WHERE  DICTATED 
COMPREHENSIVE  MANUFACTURING  PLAN  DEVELOPED 
PRODUCTION  SCHEDULES  COMPATIBLE  WITH  DELIVERY  REQUIREMENTS 
VALUE  ENGINEERING  AND  QUALITY  PROGRAMS  DEVELOPED 
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PLANNING  MICOM'S  PRODUCTION  READINESS  REVIEWS 


HAROLD  G.  PEACOCK 
US  Army  Missile  Command 
Redstone  Arsenal,  AL  35898 


ABSTRACT 

Production  planning  must  take  place  during  Engineering  Development  (ED).  The 
Production  Readiness  Review  process  has  been  developed  to  assure  that  this  planning 
takes  place  and  to  identify  weaknesses  and  problems  in  the  production  planning.  Planning 
the  PRR  is  just  as  critical  as  planning  for  production.  Areas  covered  in  planning  the 
Missile  Command's  Production  Readiness  Reviews  include  selecting  team  personnel, 
training  personnel,  developing  team  work  packages,  structuring  team  activity,  and,  finally, 
establishing  a  communication  network.  This  planning  effort  includes  coordinating  all 
activities  with  the  supporting  contractors  and  major  suppliers. 

I.  INTRODUCTION 

A.  Background.  Production  Readiness  Reviews  are  required  by  DoD  Directive 
5000.34  and  AR  70-67  with  instructions  provided  by  DoD  Instruction  5000.38.  MICOM 
Regulation  70-33  places  the  MICOM  focal  point  for  conducting  PRRs  in  the  Production 
Engineering  Division  -  a  division  within  the  System  Engineering  Directorate.  This  division 
usually  provides  the  PRR  chairman,  PRR  coordinator,  and  Producibility  Engineering  and 
Planning  (PEP)  panel  leader  and  some  PEP  members.  The  remaining  panel  leaders  and 
team  members  come  from  the  other  MICOM  directorates,  such  as  Product  Assurance, 
Guidance  and  Control,  Propulsion,  and  other  directorates  as  required.  Obtaining 
assistance  from  the  various  MICOM  directorates  requires  support  at  the  Command  level 
and  at  the  Project  Management  level.  MICOM  has  made  the  commitment  and  enforces  its 
policy  through  MICOMR  70-33  which  formalizes  a  total  commitment  to  DoD  Directive 
5000.34  and  AR  70-67. 

B.  General.  The  PRR  usually  starts  in  the  early  stages  of  Engineering 
Development  (ED)  and  reviews  the  producibility  and  production  planning  which  must  begin 
early  in  order  that  technical  and  economic  considerations  can  be  given  to  the  product 
manufactured.  As  the  product  development  progressively  matures,  the  development  tasks 
are  identified  and  planned  for  PRR  review  such  as  design  finalization  and  verification, 
parts  identification  and  qualification,  and  producibility  and  manufacturing  technology 
assessments.  Later,  selection  and  design  tasks  are  identified  in  manufacturing  areas  such 
as  equipment,  tooling,  test  equipment,  plant  layout,  design  maturity,  tool  qualification, 
and  logistic  considerations.  Still  later,  production  tasks  are  accomplished  such  as 
developing  the  process  paper,  proofing  specific  manufacturing  processes  and  equipment, 
initial  production  facilities  development,  manufacturing  information  systems  and  reports 
identification,  and  long  lead  procurement  planning.  Planning  for  the  PRR,  therefore,  is 
very  critical  since  the  purpose  of  the  PRR  is  to  insure  planning  for  production  is  complete. 

II.  PLANNING  MICOM  PRODUCTION  READINESS  REVIEWS  (PRRs) 

A.  Project  PRR  Plan.  The  first  step  in  planning  the  MICOM  PRR  is  the 
preparation  of  the  basic  requirements  for  a  project  PRR.  This  plan  is  prepared  by  the 
PRR  chairman,  coordinator,  and  staff  personnel  of  the  Production  Engineering  Division, 
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whose  responsibility  is  to  provide  production  engineering  services  as  technical  agents  to 
the  Program  Manager.  The  formal  plan  is  required  by  AR  70-67  sixty  days  prior  to  the 
first  or  Initial  Production  Readiness  Review  (IPRR).  As  a  minimum,  this  plan  identifies 
the  product,  the  contractors  and  suppliers  where  a  review  will  be  performed,  and  a 
tentative  schedule  for  these  reviews.  Special  or  unique  requirements  are  also  identified. 
..The  plan  does  not  inhibit  adding  or  excluding  a  subcontractor  or  supplier  to  the  schedule. 
Next,  the  plan  is  approved  by  the  Product  Project  Manager.  The  plan  is  then  submitted  to 
APESO  at  DARCOM  for  review.  Lastly,  the  plan  is  coordinated  with  the  contractors  for 
their  information  and  contribution. 

The  plan  is  coordinated  depending  upon  the  product,  contractors,  and  the 
existing  contract.  On  one  occasion  it  was  necessary  to  scope  a  task  and  contract  for  the 
required  PRR  contractor  support.  This  example  proved  to  achieve  a  more  cost  effective 
and  efficient  review.  PRR  team  size  was  restricted  to  approximately  12  members  in  four 
panels.  Production  documentation  was  limited,  as  it  should  be,  to  existing  data. 
Contractor  personnel  involvement  was  to  transmit  information  directly  rather  than  to 
prepare  extensive  briefings  with  a  significant  number  of  "dry  runs"  and  top  level 
massaging.  It  was  brief,  to  the  point,  and  more  representative  of  existing  conditions. 

B.  Team  Personnel.  The  next  planning  task  which  must  be  accomplished  is 
personnel  selection.  The  Production  Engineering  Division  coordinates  the  selections  of 
team  members  and  obtains  agreements  from  the  members  and  their  supervisors  prior  to 
requesting  personnel  assignment  by  members  and  their  supervisors  prior  to  requesting 
personnel  assignment  by  the  MICOM  Chief  of  Staff.  Individuals  are  selected  based  upon 
their  potential  to  contribute  to  an  independently  and  objectively  conducted  review.  Team 
personnel  are  chosen  after  evaluating  the  following  personal  factors:  experience, 
education,  communciation  skill,  personality  traits,  and  job  position.  Therefore,  member 
candidates  are  identified  based  upon  these  specific  elements,  the  product  and  upon  the 
contractor/facilities  involved. 

1.  Experience.  The  team  experience  must  be  given  the  proper  mix,  e.g., 
propulsion,  explosives,  safety,  guidance  and  control,  logisitics  and  manufacturing. 

2.  Education.  Education  provides  technical  knowledge  which  allows 
overlapping  in.  technical  areas  where  a  fresh  viewpoint  can  be  beneficially  expressed. 
Expertise  is  based  upon  technical  training  as  well  as  experience  and  both  are  necessary  to 
establish  creditability. 

3.  Communication  skills.  Each  member  will  be  required  to  exchange 
information,  the  team  members  formulating  a  line  of  questioning  that  will  extract  desired 
information  while  maintaining  a  pleasant  contractor  working  relationship.  An  open 
discussion  environment  is  critical;  neither  party  should  "play  games."  The  reviewer  must 
know  what  is  important  and  how  to  express  it. 

4.  Personality  Traits.  The  personality  traits  include  attitude,  confidence  in 
ability,  dress,  and  cooperativeness.  These  traits  will  determine  the  effectiveness  of 
implementing  education  and  experience  characteristics.  Let  us  examine  these  traits  one 
by  one. 


a.  If  one's  attitude  toward  either  the  PRR,  program,  or  contractor  is 
not  one  of  willingness  to  constructively  contribute,  then  that  individual  should  be  excluded 
from  serving  on  the  PRR  team. 

b.  Confidence  is  very  important.  Adequate  planning,  training,  and 
experience  in  these  reviews  will  increase  confidence  in  one's  ability. 
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c.  First  impressions  are  important  in  establishing  each  member's 
status  in  the  eyes  of  others.  It  is  important  that  the  contractor  personnel  know  that  they 
are  briefing  "key"  and  "important"  personnel;  proper  dress  enhances  this  impression. 

d.  Cooperation  with  other  team  members  and  contractor  personnel  is 
needed  to  insure  successful  review.  Exchanges  of  observation,  clarification  of  objectives 
and  procedures,  and  willingness  to  take  on  unpleasant  tasks  are  necessary  for  a  beneficial 
Production  Readiness  Review.  The  team  is  staffed  from  the  various  directorates  at 
MICOM  which  specialize  in  the  major  technologies  required  to  develop  Army  missile 
systems.  Once  the  potential  PRR  members  have  been  identified,  they  and  their 
supervisors  are  contacted.  Tentative  PRR  schedules  are  proposed  and  coordinated. 

C.  Team  Training.  Training  team  members  takes  the  form  of  conducting  several 
meetings  where  experienced  PRR  participants  speak  on  review  "how  to"  techniques, 
lessons  learned,  experiences  with  the  manufacturer  to  be  visited,  when  possible,  and 
briefing  by  the  laboratory  and  project  office  personnel.  These  meetings  are  scheduled  to 
provide  time  for  detail  discussion  by  the  prospective  PRR  team  members.  The  meetings 
are  used  to  explain  the  PRR  working  techniques,  schedules,  working  packages,  and 
reporting/writing.  Many  members  have  never  served  on  a  PRR  team.  Engineers  are  asked 
to  become  investigative  analysts,  a  role  new  to  most  engineers;  however,  they  must  have 
the  courage  and  confidence  for  face-to-face  confrontation  -  many  people  the  PRR 
members  confronted  will  become  emotional,  others  will  not  answer  the  question  you  think 
you  are  asking  (they  use  a  play  on  words);  therefore,  you  must  make  sure  you  understand 
the  answer  given.  Ask  for  proof,  repeat  their  answer  the  way  you  understand  it,  record 
significant  findings  and  the  name  of  the  company  representative  giving  the  information. 
Know  your  product,  your  company,  and  the  information  desired.  Always  place  the 
responsibility  to  demonstrate  readiness  upon  the  contractor.  The  purpose  of  training  is  to 
inform  each  team  member  on  what  to  expect.  It  must  also  cover  techniques  and  use  of 
PRR  forms  if  PRR  goals  are  to  be  adequately  accomplished.  Training  must  let  the  team 
member  know  what  is  required  of  him  technically  and  in  proper  personal  conduct.  Nothing 
destroys  respect  for  the  team  quicker  than  a  member  being  inattentive  during  a 
contractor  briefing  — get  up  and  take  a  break  or  take  notes.  Chances  are  the  notes  will  be 
useful  in  documenting  a  finding  or  will  start  the  investigator  on  the  tail  of  a  concern. 

Training  the  team  is  the  most  positive  way  to  assure  an  efficient  PRR.  In 
addition  to  building  self-confidence,  training  instills  cooperation  within  the  team  including 
contractor  personnel.  Training  also  improves  the  level  or  value  of  recommendations  from 
the  team.  Of  course,  training  does  not  give  technical  expertise  but  it  does  show  the 
capability  to  implement  the  expertise. 

D.  Work  Packages.  Work  packages  are  developed  from  a  standard  MICOM  PRR 
instruction.  The  purpose  of  the  work  package  is  to  make  it  easier  to  finalize  the  findings 
by  developing  a  standard  format  for  all  panels  and  members.  These  forms  will  vary  from 
project  to  project  but  will  remain  standard  with  a  specified  project.  This  instruction 
defines  PRR  objectives,  establishes  PRR  structure,  and  sets  general  scopes  and  limits  in 
each  functional  area.  Each  functional  area  is  then  tailored  to  the  specific  product, 
contractors,  and  reviewing  team.  The  tailoring  information  generally  comes  from 
technical  reviews,  reports,  and  briefings  on  the  project.  Other  information  sources 
include  the  DCP,  RFP,  and  the  development  plan.  The  work  packages  are  provided  the 
members  at  an  early  training  session  and  the  panel  leader  is  requested  to  suggest  tailoring 
by  adding  special  interest  areas. 

Working  forms  such  as  Request  for  Information,  Areas  of  Concerns  (AOC), 
Unsatisfactory  Findings  (UF),  and  panel  report  outlines  are  tailored  to  be  compatible  with 
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the  production  development  and  PRR  review  status.  For  instance,  the  AOC  form  is  used 
in  the  first  stages  and  changed  to  an  unsatisfactory  finding  form  in  the  last  stages.  The 
use  and  purpose  of  each  form  is  discussed  thoroughly  during  the  training  sessions. 

E.  Structure.  Perhaps  structuring  the  PRR  is  the  most  important  area  of  planning 
production  review.  Structuring  the  review  effort  involves  the  following  factors: 
scheduled  events,  job  descriptions,  resources,  and  travel  requirements.  First,  scheduling 
the  events  serves  to  plan  each  day  in  detail  from  the  daily  start-up  meeting  to  the 
afternoon  close-out  meeting.  The  start-up  meeting  serves  to  provide  general  administra¬ 
tive  details  and  assures  members  are  present.  The  close-out  meeting  must  include  the 
panel  leaders  and  on  small  teams  or  where  there  is  confusion,  all  the  team  members  should 
be  invited.  This  government-only  meeting  assures  cooperation  and  information  exchange, 
including  problem  discussions.  After  the  government-only  meeting,  the  PRR  Chairman 
and  coordinator  should  meet  with  the  contractor  representative  to  establish  splinter  or 
panel  meetings,  clarify  information  needs,  and  solve  any  problems  between  the  PRR  team 
member  and  contractor  personnel.  Second,  a  job  description  does  not  necessarily  need  to 
be  formal;  however,  it  is  necessary  for  each  member  to  clearly  understand  what  is 
expected  of  them.  This  understanding  must  include  daily  activity  toward  their  area  of 
responsibility  and  to  other  panels,  interaction  in  the  PRR  team,  and  in  reporting  areas  of 
concern  and  final  report.  Third,  available  resources  must  be  planned  and  are  necessary  to 
complete  an  effective  review.  These  resources  include  areas  for  panel  meetings,  existing 
data,  a  listing  of  data  in  each  subject  area,  and  contractor  support  or  one-on-one  or  panel 
discussion.  Ask  the  ^contractor  for  an  overview  of  listing  of  acronyms  and  briefing  vu- 
graphs  in  advance,  rourth,  identify  the  choices  of  travel,  the  motels  and  their  cost,  and 
try  to  identify  personnel  for  car  sharing  purposes.  Generate  a  personnel  list  and  provide 
adequate  time  for  introductions  and  getting  acquainted  during  the  training  meetings. 
Remember,  on-site  overview  and  other  team  meetings  must  be  designed  to  optimize  the 
effort  while  leaving  ample  time  for  individual  and  panel  activity.  The  structured  agenda 
aids  in  eliminating  confusion,  developing  team  unity,  and  identifying  data  requested  by  any 
member.  The  structured  agenda  can  mean  the  difference  between  success  or  failure  with 
a  reluctant  contractor. 

F.  Communication.  Reports  are  an  important  means  of  communication  which  is 
the  major  element  in  successful  Production  Readiness  Review  planning.  The  communica¬ 
tion  network  must  consider  inter-team  communication,  PRR  team  to  contractor  interface 
and  the  formal  report.  The  daily  close-out  meetings  give  team  structure  and  provide  a 
means  for  coordinating  action  items  and  provide  daily  progress  reports  to  the  contractor. 
The  contractor  is  normally  very  interested  and  concerned  in  the  PRR  findings  and  the 
possible  action  which  may  result.  Therefore,  he  should  be  kept  thoroughly  informed  of  the 
findings  and  conclusions  and  know  the  desired  procedure  for  corrective  and  close-out 
action.  All  PRR  members  must  always  remember  in  their  discussions  with  the  contractor 
that  a  government  project  is  a  two-party  program  and  both  parties  have  an  investment: 
the  government  in  money,  R&D  effort  and  mission  requirement;  the  contractor  in  millions 
of  dollars  invested  in  capital  equipment  and  facilities  dedicated  to  the  program. 

Contractor  coordination  and  participation  must  be  established  very  early  in  the 
planning  stage.  Good  communications  will  assure  proper  reception  and  disposition  of  PRR 
concerns  requiring  his  action.  The  contractor  is  a  part  of  the  PRR  team.  He  must  be 
made  to  feel  that  he  is  a  contributing  party  in  this  review,  which  will  improve  the 
acceptance  status  of  the  program;  therefore,  include  the  contractor  in  scheduling  on-site 
visits,  discussing  team  member  conclusions  and  recommendations,  follow-up  actions,  and 
schedules  and  management  levels  of  the  PRR  reports. 
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III.  SUMMARY 


Planning  is  critical  in  preparation  for  production  and  must  start  during  the  early 
stages  of  Engineering  Development.  The  Production  Readiness  Review  (PRR)  process  has 
been  developed  to  assure  that  this  planning  takes  place  and  to  identify  weaknesses  and 
problems  in  the  production  planning.  Planning  is  just  as  critical  when  preparing  for 
Production  Readiness  Reviews  as  when  planning  for  production.  The  importance  of 
planning  PRRs  is  recognized  when  one  realizes  that  the  PRR  is  composed  of  experts  with 
different  backgrounds  who  find  themselves  working  together  for  the  first  item. 
Therefore,  planning  the  PRR  must  be  accomplished  thoroughly  and  with  great  care  to 
insure  that  the  team  members  will  work  together,  overlapping  in  technical  and 
management  areas  and  at  the  same  time  maintaining  their  independence  and  objectivity. 
Planning  the  PRR  covers  many  facets  including  the  following:  (1)  Team  Personnel:  The 
team  is  organized  into  functional  panels  and  orders  are  issued  at  the  Command  level. 
Team  members  are  selected  from  the  many  independent  directorates  which  cover  the 
various  technologies  in  missile  system  research  and  development.  (2)  Training:  Training 
covers  review  techniques,  lessons  learned,  manufacturer  history,  and  briefing  project 
personnel.  (3)  Work  Packages:  During  this  stage  of  planning,  work  packages  are  developed 
which  provide  working  sheets  and  "starter"  or  "fail-back"  questions  of  the  assigned  review 
areas.  Basic  elements  of  the  work  package  are  taken  from  standard  procedures  developed 
by  the  Production  Engineering  Division  of  MICOM.  (4)  Structure:  Each  day  is  planned 
from  the  morning  PRR  meeting  to  the  afternoon  meetings  with  panel  leaders  and  later 
meetings  with  contractor  representatives.  Significant  time  is  scheduled  for  panel 
meetings  and  individual  discussions.  (5)  Communication:  Finally,  a  plan  must  incorporate 
a  communication  network.  Planning  communication  covers  the  exit  briefing,  contractor 
follow-up,  reporting  the  findings  to  the  Project  Manager,  and  the  outline  to  be  used  for 
the  formal  report. 


103 


104 


NAVY  PRR  OVERVIEW 


William  D.  Oaks 
Navy  PESO 


Just  a  little  bit  of  history  concerning  PRRs  in  the  Navy.  The  earliest  record  goes 
back  to  the  establishment  of  the  NPESO,  originally  part  of  the  DPESO  group,  in  the  first 
Dart  of  1977.  A  primary  responsibility  of  NPESO  was  to  assure  that  all  major  weapon 
systems  being  developed  under  the  management  cognizance  of  the  Navy  would  be  sub¬ 
jected  to  a  dedicated  review  process  of  sufficient  depth  to  accurately  determine  the 
degree  of  readiness  for  production.  It  was  soon  found  that  the  review  process  itself  cou  o 
lead  to  readiness  if  conducted  as  a  time  phased  consultation  service  type  of  Pr°gram. 
^Tof  the  first  things  our  office  accomplished  was  our  own  orientation  with  the  Navy  s 
organizations  to  learn  just  what  expertise  was  available,  and  where  it  was  located.  We 
contacted  weapon  centers,  support  activities,  ordnance  stations,  analysis  centers,  etc., 
and  were  pleasantly  surprised  to  learn  that  there  is  an  enormo_us  amount  of  technical 
talent  available  in  the  field.  However,  this  talent  had  never  been  coordinated  Navy-wide 
(and  we've  still  only  scratched  the  surface  in  this  respect).  This  talent  has  been  assembled 
around  some  project  requirements  and  field  activities  requirements  but  it  had  never  been 
used  Navy-wide  -  the  Navy's  composed  of  a  series  of  rather  autonomous 
very  few  groups  within  the  Navy  have  responsibilities  across  the  board.  HQNAVMA 

one  of  those  few. 

So  shortly  after  our  formation  in  April,  we  got  our  first  assignment  -  to  conduct  a 
PRR  on  the  CH-53E  Super  Stallion,  heavy  lift  helicopter  being  developed  by  Sikorsky 
Aircraft.  The  planning  began  -  we  got  the  help  of  NAVWESA  located  at  the  Navy  Yard 
borrowed  one  additional  man  from  the  DPESO  organization,  and  conducted  the  first  Navy 
PRR  in  October,  just  three  weeks  before  the  DoD  Directive  5000.34  was  signed.  ven 
though  we  were  still  technically  part  of  DPESO,  we  all  put  on  Navy  hats  and  our  boss, 
Wayne  Crispen,  functioned  as  the  Team  Leader,  reporting  directly  to  the  Project  Manager 
for  all  matters  related  to  the  PRR.  The  final  report  was  included  as  an  appendix  to  the 
DCP.  We  participated  in  a  few  other  reviews,  the  PHALANX,  20mm  Gun;  AEGIS  System, 
and  8"  Major  Caliber  Lightweight  Gun  System;  but  these  were  DoD  reviews,  and  not  run  by 

the  Navy. 

We  became  part  of  the  Navy  in  April  1978,  but  we  were  still  co-located  with  DPESO 
organization  until  early  1979. 

July  1978,  however,  the  Navy  started  PRRs  in  a  relatively  big  way.  The  F/A-18 
program,  led  by  the  Naval  Air  Systems  Command;  and  the  Joint  Cruise  Missile  Program 
led  by  the  JCM  Project  Office  -  you'll  hear  much  more  about  these  two  programs  from 
their  respective  team  leaders  a  little  later  in  the  program. 

So  far  to  date,  the  Navy  has  conducted  87  reviews  of  56  contractors  involving  11 
projects. 

However,  as  you  might  suspect,  most  of  the  reviews  have  been  conducted  on  just  a 
few  projects.  In  fact,  three  projects  -  the  F-18,  Cruise  Missile,  and,  a  new  one  -  the 
LAMPS  III,  represent  78  percent  of  the  reviews  held.  The  LAMPS,  by  the  way,  is  the  Light 
Airborned  Multi-Purpose  System,  which  is  basically  a  helicopter/shipboard/electronics 
package  involved  primarily  in  anti-submarine  warfare.  It  is  q  joint  NAVAIR/NAVSEA 
•  project,  with  NAVAIR  leading  the  PRR  effort,  and  it's  similar  in  magnitude  to  the  F-18 
and  Cruise  Missile  Programs. 
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The  size  of  the  Navy  PRR  team  has  ranged  all  the  way  from  six  to  44  Government 
participants  —  we  have  tried  to  key  the  size  of  the  team  to  criticality,  complexity,  and 
the  size  of  the  contractor  organization  to  be  reviewed.  Actually,  with  an  experienced 
group  of  professionals,  the  size  of  the  team  can  be  determined  by  the  number  of 
functional  area/panel  meetings,  since  it's  advisable  to  have  at  least  two  reviewers  in  each 
meeting,  or  sub-panel  meeting.  This  gives  you  two  judgements  of  the  contractor's 
responses  to  the  interview  questions. 

LESSONS  LEARNED 


Do  your  homework  —  assemble  all  significant  paperwork  for  review  by  the  team  to 
help  any  new  people,  that  are  new  to  the  project,  to  become  familiar  with  what  has  been 
done  what's  intended,  etc.  —  the  production  plan,  deviations  and  waivers  requested, 
ECPs,  significant  test  results,  the  DCP,  any  previous  "ad  hoc  committee"  reports,  and  the 
like. 


Don't  get  pushed  into  a  too  tight  schedule  on-site.  Allow  a  week  if  you've  got  a 
week's  work  to  do. 

Don't  go  in  with  an  army  unless  you're  really  sure  from  previous  planning  that  you're 
going  to  need  it.  A  little  bit  of  overskill  goes  a  long  way. 

Be  sure  to  cover  all  significant  branch  operations.  An  outfit  like  TI,  for  instance, 
has  machining  operations  assembly  and  test  fabrication  of  PWBs,  etc.,  spread  over  several 
counties  of  Texas  and  on  some  projects,  they  may  all  contribute  to  the  success  of  the 
project. 

The  Navy  PRR  teams  spend  a  good  bit  of  effort  in  the  selection  of  appropriate 
subcontractors,  and  you'll  hear  about  this  in  more  detail  from  some  of  our  other 
presenters. 

It  has  become  more  and  more  obvious  lately  that  some  sort  of  documentation 
system,  or  data  bank,  needs  to  be  created  on  a  DoD-wide  basis  to  allow  the  dissemination 
of  all  these  ."lessons  learned."  This  conference  is  an  example,  and  it  should  help,  but  it 
won't  answer  all  the  questions  I've  had  in  the  last  few  months.  I've  had  project  people  call 
me  and  want  to  know:  "What  are  the  soft  points  with  Contractor  X?",  "What's  the  quality 
philosophy  of  Contractor  Y?",  "What  kind  of  importance  does  this  guy  associate  with 
production  planning?"  The  quality  world  is  in  the  process  now  of  creating  a  data  bank  - 
joint  with  all  Services,  since  they  have  a  requirement  to  avoid  awarding  contracts  to 
sources  with  known  "poor  track  records."  Obviously,  there  are  enormous  problems  —  legal 
—  for  example,  you  can't  create  "blacklists"  and  political  since  most  PRRs  tend  to  show 
the  project  in  a  poor  light,  but  evidently  there  is  a  bona  fide  need  for  something.  All  I  can 
say  at  this  time  is  to  think  about  it.  Maybe  you  can  make  some  recommendations  at  the 
wrap-up. 

Currently,  we  have  five  projects  scheduled  for  additional  PRRs  and  several  others 
that  could  be  scheduled  for  near-term  reviews.  The  LAMPS,  as  I  mentioned  before,  is  our 
largest  on-going  project  with  some  25  reviews  scheduled  and  critical  second  or  third  tier 
subcontractor  reviews  scheduled  as  the  need  becomes  apparent. 

The  HARM,  a  high  speed  anti-radiation  missile,  joint  with  the  Air  Force,  still  has  the 
reviews  to  support  Milestone  III  remaining. 

The  F-18  is  essentially  complete.  However,  full  rate  production  has  not  been 
achieved,  and  there  could  be  some  follow-up  required. 
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PRRs  for  some  variants  of  the  Joint  Cruise  Missile  are  complete  and  others  are  on¬ 
going.  PIFRRs  are  planning  for  rate  reviews  held  after  Milestone  III  —  you'll  hear  more 
about  these  later. 

The  Sidewinder  9M  variant,  going  on  this  week,  is  not  a  major,  but  certainly  is 
critical. 

In  the  future  —  of  course  we're  going  to  continue  the  LAMPS  and  HARM  reviews 
much  as  they  are  now  scheduled  —  looking  at  a  late  '81  DSARC. 

There  are  new  variants  of  the  Cruise  Missile  -  specifically,  the  sea-launched  type. 

The  SPARROW  -  7M. 

The  AV-8B  V/STOL  Aircraft  -  in  a  holding  mode  right  now  --  involves  interface 
problems  with  British  Aerospace  and  Rolls  Royce. 

A  couple  of  new  aircraft. 

And  we've  started  to  look  closely  at  the  ship  construction  area.  The  work  so  far  is 
just  looking  at  the  typical  functional  areas  reviewed  in  any  other  type  project  and  then 
relating  these  by  a  comparison  matrix  to  the  sometimes  unique  problems  of  ship 
construction.  It's  being  done  element  by  element  using  the  guidelines  of  DoD  Instruction 
5000.38.  The  intent  is  to  develop  the  process  into  a  true  time-phased  study  integrated 
into  the  total  system  of  reviews  typical  of  ship  construction.  Two  differences  became 
immediately  apparent.  First,  ship  construction  is  an  extremely  long  cycle  —  four  or  five 
years.  Second,  it  involves  very  low  rates.  As  a  result,  tooling,  as  it  is  usually  thought  of  - 
-  that  is,  unique  dies,  and  cutting  tools,  etc.,  are  almost  non-existent  in  the  yards  —  tools 
are  general  purpose  shop  tools,  like  drill  presses,  welders,  and  the  like.  Facilities  and 
material  handling  constraints  like  crane  coverage  and  capacity  become  major  issues,  and 
since  it's  done  outside,  weather  is  also  a  constraining  factor.  By  and  large,  though,  the 
progress  has  been  encouraging. 

The  Navy  is  learning  the  advantages  of  the  PRR  process  and  we're  applying  it  to 
more  and  more  "other  than  major"  programs.  We  will  continue  in  this  direction  until  we 
find  that  the  cost  of  implementation  overshadows  the  returns. 

We  are  investigating  the  application  of  PRR  technology  earlier  in  the  project  ~ 
maybe  as  part  of  the  source  selection  process. 

Naval  Material  Command  has  established  a  computer  data  bank  to  be  used  as  a 
repository  for  all  project  review  data  —  the  PRDRs  (Preproduction  Reliability  Design 
Reviews),  PRRs,  quality  reviews,  etc.  The  data  are  being  compiled  as  coded  action  items. 
Later,  it  will  be  possible  to  display  data  by  contractor,  or  functional  area,  or  project,  or 
any  combination. 

We've  been  trying  for  some  time  to  combine  parts  of  reviews  and  eliminate  what 
seems,  on  the  surface  at  least,  to  be  duplication.  The  comparison  matrix  being  used  now 
by  NAVSEA  may  help  in  the  analysis. 

That  just  about  wraps  it  up  for  the  general  overview.  Now  we  will  get  into  more  of 
the  details. 
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PRR  HISTORY  IN  THE  NAVY 


®  ESTABLISHMENT  OF  SERVICE  LEVEL  (NAVY)  PESO  -  APRIL,  1977 
®  1ST  PRR  RUN  BY  NAVY  -  1ST  WEEK  OF  OCTOBER,  1977 
®  DOD  DIRECTIVE  5000.34  -  "DOD  COMPONENTS— RESPONSIBLE  FOR— PPRs' 
®  NAVY  PESO  PART  OF  NAVY  -  APRIL,  1978 
®  TO  DATE  -  87  REVIEWS  OF  56  CONTRACTORS 
®  11  PROJECTS  -  3  REPRESENT  78%  OF  REVIEWS 
®  ATTENDANCE  -  44  TO  6  GOVERNMENT  PARTICIPANTS 
®  LESSONS  LEARNED 

OPTIMUM  ATTENDANCE 

ENOUGH  TIME 

BRANCH  OPERATIONS 

SUBCONTRACTORS 

NEED  FOR  OOO  OATA  BANK 

HOMEWORK 


CURRENT  PRRs 

LAMPS  -  25  REVIEWS  SCHEDULED 
HARM  -  3  REVIEWS  REMAIN 
F-18  -  FOLLOW-UP  ONLY 

JOINT  CRUISE  MISSILE  -  9  PRRs  REMAIN, 
PLUS  6  PFRRs 

SIDEWINDER  -  9M 
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FUTURE  OF  PRR's  IN  THE  NAVY 


©  CONTINUE  LAMPS  &  HARM 
©  NEW  VARIANTS  OF  CRUISE  MISSILE 
©  SPARROW  -7M 
©  AV-8B  V/STOL  AIRCRAFT 
©  OTHER  NEW  AIRCRAFT 
©  SHIP  CONSTRUCTION 

©  MORE  APPLICATIONS  OF  PRR's  TO  "OTHER  THAN  MAJOR" 
PROJECTS 

©  EARLIER  APPLICATION 
©  CORPORATE  MEMORY 
©  COMBINATIONS  OF  REVIEW 
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LAWRENCE  STAKEM 


1  AH  HERE  TO  PRESENT  THE  NAVAL  AIR  SYSTEMS  COMMAND  PRODUCTION  READINESS 
REVIEW  PERSPECTIVE  AND  APPROACH  AS  VIEWED  BY  THE  NAVAL  WEAPONS  ENGINEER¬ 
ING  SUPPORT  ACTIVITY  WHICH  PROVIDES  THE  MAJOR  PRODUCTION  RELATED  SUPPORT 
TO  NAVAIR  ON  PRR's. 


Ill 


INTRODUCTION 


'•  OVERVIEW  OF  NAVY  PRR  PROCESS 

II.  NAVY  PRR  PHASES 

III.  DYNAMICS  OF  PRR  PROCESS 

IV.  SUMMARY  OF  PRR  LESSONS  LEARNED 


THESE  ARE  THE  FOUR  MAJOR  TOPICS  WHICH  I  WILL  ADDRESS  DURING  THIS  BRIEFING- 

THE  MAJOR  EMPHASIS  WILL  BE  ON  THE  NAVY  PRR  PHASES  AND  ESPECIALLY  THE 
PLANNING  PHASE  PLUS  THE  DYNAMICS  OF  THE  PRR  PROCESS- 
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OVERVIEW  OF  NAVY  PRR  PROCESS 


•  PRR  POLICY  AND  GUIDANCE 

•  NAVY  PRR  PROCESS 

•  DOD  AND  NAVY  ORGANIZATIONAL 
RELATIONSHIPS 

•  NAVY  PRR  ORGANIZATION 


THE  NAVY  METHOD  OF  PERFORMING  PRODUCTION  READINESS  REVIEWS  IS 
CONFORMANCE  WITH  THAT  OF  THE  DEPARTMENT  OF  DEFENSE- 


FOUR  (A)  SUBSEQUENT  SLIDES  WILL  CONFIRM  THIS  STATEMENT- 


STRICT 
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PRODUCTION  READINESS  REVIEW 
POLICY  AND  GUIDANCE 


DODD  5000.1 
DODD  5000.34 
D0DI  5000.2 
D0DI  5000.38 
SECNAVINST  4801.1 
NAVMATINST  4801.1 
NAVAIRINST  (DRAFT) 


MAJOR  SYSTEM  ACQUISITIONS,  19  MARCH  1980 

DEFENSE  PRODUCTION  MANAGEMENT,  31  OCTOBER  1977 

MAJOR  SYSTEM  ACQUISITION  PROCEDURES,  19  MARCH  1980 

PRODUCTION  READINESS  REVIEWS,  24  JANUARY  1979 

DEFENSE  PRODUCTION  MANAGEMENT,  5  MARCH  1979 

DEFENSE  PRODUCTION  MANAGEMENT,  28  MAY  1980 

PRODUCTION  READINESS  REVIEWS:  POLICY, 
RESPONSIBILITIES  AND  PROCEDURES 


THIS  IS  A  LIST  OF  THE  MAJOR  DEPARTMENT  OF  DEFENSE  AND  NAVY  GUIDANCE 
DOCUMENTS  USED  BY  US  TO  PLAN,  IMPLEMENT  AND  COMPLETE  THE  NAVY  PRR 
PROCESS  PRIOR  TO  DSARC- 


IT  SHOULD  BE  NOTED  THAT  NAVMATINST  4801-1  HAS  JUST  BEEN  REVISED  ON  4 
NOVEMBER  80--THE  PRESIDENTIAL  ELECTION  DATE- 
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PRR  DOD  AND  NAVY 
ORGANIZATIONAL  RELATIONSHIPS 


COORDINATION  AND  GUIDANCE 


WE  THE  NAVY.  RECOGNIZE  THE  REQUIREMENT  THAT  THE  DEPARTMENT  OF  DEFENSE 
PRODUCT  ENGINEERING  SERVICES  OFFICE  (DPESO)  MUST  CONDUCT  THEIR  OWN 
INDEPENDENT  PRR  ASSESSMENT.  IN  AN  EFFORT  TO  SAVE  COST.  TIME.  MINIMIZE 
DISRUPTIONS  TO  THE  CONTRACTOR  AND  PROVIDE  ON-SITE  GUIDANCE  WE  HAVE  HAD 
MEMBERS  OF  DPESO  AND  NPESO  ACCOMPANY  THE  NAVY  PRR  TEAM  ON  NEARLY  ALL  PRRs 
WE  HAVE  PERFORMED. 


THE  NAVY  CONDUCTS  PRRs  ON  ALL  PROJECTS  AND  SELECTED  PROGRAMS-  OUT 
DEFINITION  OF  A  PROJECT  IS  A  MAJOR  WEAPON  SYSTEM  SUCH  AS  THE  F/A-18 
HORNET.  THE  F-W  TOMCAT  AND  OTHERS-  A  PROGRAM  IN  THE  NAVY  VERNACULAR  IS 
A  MAJOR  PIECE  OF  GFE  SUCH  AS  AN  ENGINE  OR  A  STANDARD  MISSION  COMPUTER 
SET- 
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NAVY  PRR  PROCESS 


•  CONCEPTS  IN  ACCORDANCE  WITH  DOD 
AND  SERVICE  DIRECTIVES 

•  NAVY  PRR  PROCESS  FORMALIZED 

•  NAVY  PRR  DEFINITION,  PURPOSE  AND 
GOAL  IDENTICAL  WITH  DOD 


HE  BELIEVE  THAT  THE  PROCEDURES  HE  USE  IN  THE  NAVY  PRR  PROCESS  ARE  IN 
TOTAL  ACCORD  HITH  THE  INTENT  OF  THE  DOD  PRR  PROGRAM  AS  DEMONSTRATED  BY 
THIS  SLIDE. 
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TYPICAL  NAVY  PRODUCTION  READINESS  REVIEW 
ORGANIZATION 


THE  NORMAL  NAVY  PROJECT  MANAGEMENT  EFFORT  IS  A  MATRIX  TYPE  ORGANIZATION. 

THE  PROJECT  OR  PROGRAM  MANAGER  HAS  THE  ULTIMATE  AUTHORITY  TO  IMPLEMENT 
THE  PRODUCTION  READINESS  REVIEW  FOR  HIS  SYSTEM-  THE  PRR  TEAM  LEADER  IS 
APPOINTED  BY  HIM  AND  IS  NORMALLY  A  MEMBER  OF  HIS  STAFF- 

THE  ASSISTANT  PRR  TEAM  LEADER  IS  NORMALLY  APPOINTED  BY  THE  CLASS  DESK- 
THE  CLASS  DESK  IN  THE  NAVY  IS  COMPRISED  OF  THE  TECHNICAL  ENGINEERING 
SPECIALISTS  RESPONSIBLE  FOR  THE  ENGINEERING  DESIGN,  DEVELOPMENT  AND  TEST 
FUNCTION. 

THE  OTHER  ORGANIZATIONS  ALSO  PROVIDE  PERSONNEL  WITH  THE  REQUIRED  EXPERTISE 
TO  CONDUCT  THE  PRR  IN  AS  COMPLETE  AND  OBJECTIVE  MANNER  AS  IS  POSSIBLE- 

I  AM  FROM  THE  NAVAL  WEAPONS  ENGINEERING  SUPPORT  ACTIVITY"  (LARRY  SHOW  ON 
CHART  YOUR  BLOCK)  AND  WE  ARE  RESPONSIBLE  FOR  ALL  PRODUCTION  RELATED 
FUNCTION  SUCH  AS  QUALITY  ASSURANCE,  PROCUREMENT,  MANUFACTURING,  PRODUCTION 
CONTROL  AND  SO  FORTH- 
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NAVY  PRR  PHASES 


•  PLANNING  PHASE 

•  IMPLEMENTATION  PHASE 

•  COMPLETION  PHASE 


THESE  ARE  THE  PHASE  OF  OUR  PRR  PROCESS  AND  AS  I  STATED  EARLIER;  THE 
PLANNING  PHASE  IS  THE  ONE  ME  CONSIDER  HOST  DYNAMIC  AND  HOST  COMPLfX- 
THEREFORE,  HOST  OF  THE  EMPHASIS  OF  THIS  BRIEFING  HILL  ADDRESS  PLANNING. 
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PLANNING  PHASE  FUNCTIONS 


•  SELECTION  OF  CANDIDATE 
CONTRACTORS 

•  PRR  TEAM  SELECTION 

•  GOVERNMENT  AND  INDUSTRY 
INDOCTRINATION  AND  COORDINATION 

•  DEVELOP  SPECIFIC  PRR 
DOCUMENTATION 


THESE  ARE  THE  FOUR  (4)  SIGNIFICANT  FUNCTIONS  TO  BE  ACCOMPLISHED  DURING 
THE  PLANNING  PHASE- 


OUR  EXPERIENCE  WITH  THE  PRR  PROCESS  IS  THAT  THE  THIRD  BULLET  'GOVERNMENT 
AND  INDUSTRY  INDOCTRINATION  AND  COORDINATION'  IS  THE  MOST  DIFFICULT  TO 
ACCOMPLISH.  THIS  IS  PRIMARILY  DUE  TO  THE  INITIAL  FEAR  OF  A  CONTRACTOR 
WHO  HAS  NEVER  HAD  ANY  EXPERIENCE  KITH  THE  PRR  PROCESS-  HE  IS  VERY 
APPREHENSIVE  AND  THE  DIFFICULTY  ARISES  IN  GETTING  HIM  TO  ACCEPT  THE 
PURPOSE  OF  THE  PRR  AND  TO  PUT  HIM  AT  EASE  SO  THAT  THE  PRR  CAN  BE  EFFEC¬ 
TIVELY  CONDUCTED  ON-SITE- 


MY  NEXT  SLIDE  MILL  LIST  SOME  OF  THE  NAVY  PROJECTS  AND  PROGRAMS  WHICH  HAD 
PRRs- 
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NAVY  PRR  PROGRAMS/PROJECTS 

PAST: 


CH-53E 

AV-8B 

HARM 

AN/AYK-14 

F/A-18 

PHOENIX 

LAMPS 

ARBS 

PRESENT  (PLANNED  TO  JUNE  81) 

LAMPS 

AN/AYK-14 

AIM-7F 

AIM-9M 

FUTURE: 

A-18 

LAMPS 

AV-8B 

HARM 

THIS  IS  ML  AN  ALL  INCLUSIVE  LIST  OF  PROGRAMS  AND  PROJECTS  FOR  FUTURE 
PRRs-  ME  ADD  TO  THIS  LIST  AS  THE  NEED  DICTATES-  FOR  EXAMPLE,  DUE  TO  THE 
IRANIAN  CRISIS  AND  THE  COMPROMISE  OF  THE  PHOENIX  MISSILE,  WE  ADDED  THE 
MODIFIED  MISSILE  TO  OUR  LIST  AND  CONDUCTED  A  PRR-  IF  SIMILAR  OR  OTHER 
CIRCUMSTANCES  REQUIRE  PRRs,  HE  PERFORM  THEM-  THIS  SURE  KEEPS  A  LOT  OF  US 
ON  THE  ROAD- 
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CONTRACTOR  SELECTION  PROCESS 


•  ANALYZE  THE  TOTAL  PROGRAM 

•  DEVELOP  A  LIST  OF  CANDIDATE 
CONTRACTORS  FOR  PRR 

I 

•  UTILIZE  SELECTION  CRITERIA 

•  TEAM  SELECT  CONTRACTORS  FOR 
GOVERNMENT  ON-SITE  PRR 

•  ADD  NEW  CONTRACTORS  AS  PRR 
PROCESS  DICTATES 


DURING  OUR  ANALYSIS  OF  THE  TOTAL  PROGRAM  TO  SELECT  CONTRACTORS  FOR  PRRs 
HE  CONSIDER  SUCH  THINGS  AS  PROGRAM  SIZE,  COMPLEXITY  AND  POINT  OF  TIME  OF 
THE  DEVELOPMENT  PHASE  AND  TIME  REMAINING  FOR  THE  DSARC* 


KEY  PERSONNEL  OF  THE  PROGRAM  MAKE  AN  INITIAL  LIST  OF  CANDIDATE  CONTRACTORS 
FOR  THE  PRR,  THIS  INITIAL  LIST  HAS  SOMETIMES  EXCEEDED  100  CONTRACTORS* 

HE  FIND  THAT  CONSTRAINTS  OF  TIME,  MANPOHER  AND  MONEY  FORCES  US  TO  REDUCE 
THIS  LIST*  THEREFORE,  HE  ESTABLISH  CRITERIA  TO  REDUCE  THIS  LIST  TO  THOSE 
CONTRACTORS  THAT  HAVE  THE  MAJOR  IMPACT  ON  THE  PROGRAM- 


THE  NEXT  SLIDE  LISTS  OUR  CONTRACTOR  SELECTION  CRITERIA* 
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CONTRACTOR  SELECTION  CRITERIA 

•  MISSION  CRITICAL 

•  RELIABILITY /MAINTAINABILITY  CRITICAL 

•  NEW  TECHNOLOGY 

•  HARDWARE  COMPLEXITY 

•  COST  CRITICAL 

•  SCHEDULE  CRITICAL 

•  SUPPORT  CRITICAL 

•  CONTRACTOR  TRACK  RECORD  AND  DEVELOPMENT 
PROGRESS 

•  WEAPON  SYSTEM  MISSION  COMPATIBILITY 
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WE  FORM  A  MATRIX  WITH  THE  CRITERIA  SHOWN  HERE  LISTED  IN  A  ROW  ACROSS  THE 
TOP  AND  THE  LONG  LIST  OF  OUR  'FIRST  CUT*  CANDIDATE  CONTRACTORS  IN  A 
COLUMN  TO  THE  LEFT • 

WE  THEN  SEND  THIS  LIST  TO  THE  KEY  ORGANIZATIONS  WHO  ARE  MOST  KNOWLEDGEABLE 
WITH  THE  PROGRAM  TO  MAKE  A  DETERMINATION  OF  THOSE  CONTRACTORS  THEY 
CONSIDER  HOST  CRITICAL  BY  PLACING  Xs  IN  THE  APPROPRIATE  BLOCKS- 

THE  TEAM  MEMBERS  GET  TOGETHER  AS  A  GROUP  TO  ANALYSE  THE  RESULTS  AND 
SELECT  THE  PRR  CONTRACTORS  FOR  ON-SITE  REVIEW- 

A  PLEASANT  SURPRISE  IS  THAT  THERE  IS  A  HIGH  DEGREE  OF  CORRELATION  AMONG 
THE  VARIOUS  PRR  ORGANIZATIONS  THAT  THIS  INDEPENDENT  JUDGMENT  GENERALLY 
SELECTS  THE  SAME  CONTRACTORS  AS  BEING  HOST  RISKY  TO  THE  PROGRAM- 

CONSTRAINTS  OF  MONEY,  TIME  AND  PERSONNEL  ARE  THEN  APPLIED  TO  NARROW  THE 
LIST  TO  A  NUMBER  OF  PRRs  THAT  ARE  FEASIBLE  TO  PERFORM- 

THE  NEXT  CHART  SHOWS  THE  RESULTS  OF  OUR  EFFORTS  AS  RELATED  TO  THE 
LIGHT  AIRBORNE  MULTI-PURPOSE  SYSTEM  (LAMPS)- 
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LAMPS 

PRODUCTION  READINESS  REVIEW  SCHEDULE 


WmRAUOU/lOCAUON 


EROImAH 


OAF,  I  N#AL ,  Toronto,  CAW 
illMIIT,  StNATFONO,  IT 

18H,  Onego,  AY 

Sperry,  Ottawa,  CAA 

Down ,  Toronto,  CAA 

Faincnilo,  Germantown,  HO 

6*E«,  Lynn,  NA 

Tl,  Dallas,  TX 

Aaytneon,  Goleta,  CA 

Ryan,  San  Ollco,  CA 

Sienna,  Buffalo,  AY 

Oowty,  Toronto,  Can 

Cleyelani  Pneunatic,  Cleveland,  OH 

Asthonautic,  Rilnaur*i,  VI 

LTV,  Dallas,  TK 

Ain  Aeieancn,  Pnoenti,  AA 

SPECO  SfRINCFIELO,  OH 

Telephonic,  Huntington,  AY 

Aatnono,  Ridoletomn,  CT 

Sperry,  Salt  Lake  City,  UT 

Sferry,  Ottana,  CAA 

Oowty,  Toronto,  CAN 

OAF  Inoal,  Toronto,  CAA 

Haniltov  Si#.  Vinoson  Locks,  Cl 

Sikorskt,  Stratfono.  Cl 

IHH,  Onego,  AY 


Rast  System 

SH-608  Aircraft 

Various  Systens 

Horiiontal  Reference  Systen 

Aast  Hvoraulics 

Control  Ino  Sit 

Engine  ( I700-GE-90I ) 

Aaoan 

Aecetver  Set,  Countermeasure 

Aavs  Raoar 

Raoto  Tenninal  Set 

Rast  Hyoraultcs 

HLG/TL6 

ALT/BAG  Indicators 
Tailcone  Assinnl* 

Sonoruoy  Launcher 
SH608  Tnansnission 
CSC/Aavn  Intef  Unit 
Magnetic  Tare  Henory 
Aaoio  Iennthal  Set 
Horizon  Reference  Systen 
Aast  Hyoraultcs 
Aast  Systen 

Elect  Flight  Cntnl  Systen 
SH-608  Aincraf i 
Various  Systems 


QATE  OF  SCHEHtlL£P_PR» 

25- 27  JUR  1980 
19-18  JUL  1980 

26  JUL  -01  AUG  1980 
15-15  AUG  1980 

26- 2B  AUG  I960 

10- 12  SEP  1980 
25-26  SEP  I960 
07-10  OCT  1980 
22-29  OCT  19B0 

27- 29  OCT  1980 
05-07  A0V  1980 
12-19  A0V  19B0 

11- 20  A0V  19B0 
02-09  DEC  1980 
09-11  DEC  1980 
06-0B  JAN  1981 

19- 16  JAA  1981 

20- 22  JAA  1981 
27-29  JAA  1981 
05-06  FEB  1981 
17-20  FEB  1981 
05-06  UAH  1981 
17-20  HAH  1981 
29-26  HAH  1981 
01-05  APH  1981 
15-17  AI*H  1981 


THIS  IS  THE  RESULT  OF  OUR  INITIAL  SUB-CONTRACTOR  SELECTION  EXCERISE  ON 
THE  LAMPS  PROJECT. 


THE  NEXT  FEN  CHARTS  HILL  ADDRESS  THE  PRR  TEAM  SELECTION  PROCESS- 
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PRR  TEAM  SELECTION 


•  CONSISTS  OF  INDIVIDUALS  HAVING  THE  REQUISITE 
EXPERTISE  AND  EXPERIENCE 

•  SELECTED  FROM: 

•  FUNCTIONAL  CODES  OF  SYSTEMS 
COMMANDS 

•  NAVY  FIELD  ACTIVITIES 

•  LOCAL  GOVERNMENT  OFFICES 

•  OTHER  SERVICES/AGENCIES 

•  CIVILIAN  INSTITUTIONS/UNIVERSITIES 

•  PRIVATE  INDUSTRY 


THERE  ARE  TNO  IMPORTANT  POINTS  RELATED  TO  TEAM  SELECTION,  THEY  ARE: 

1)  WE  ALWAYS  DESIRE  THE  HOST  EXPERT  PERSONNEL  BUT  SOMETIMES  THEIR 
AVAILABILITY  OR  NON-AVAILABILITY  FORCES  US  TO  GO  WITH  THE 
NEXT  BEST-  THIS  IS  AN  INHERENT  MATRIX  ORGANIZATIONAL  PROBLEM- 

2)  WE  GO  TO  ANY  KNOWN  SOURCE  TO  SELECT  QUALIFIED  PEOPLE- 
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TYPICAL  FUNCTIONAL  AREAS 


•  PROGRAM  MANAGEMENT 

•  PRODUCTION  METHODS  AND  PROCESSES 

•  ENGINEERING  PRODUCT/DESIGN 

•  PRODUCTION  CONTROL 

•  PRODUCTION  COST 

•  FACILITIES  AND  EQUIPMENT 

•  MAKE-OP-BUY 

•  TOOLING  AND  TEST  EQUIPMENT 

•  TESTING  STATUS 

•  MATERIAL 

•  SUBCONTRACTING 

•  MANPOWER 

•  PROCUREMENT 

•  PLANT  LAYOUT 

•  MANUFACTURING  PLANNING 

•  INTEGRATED  LOGISTICS  SUPPORT 

•  SOFTWARE 

TO  ACHIEVE  GREATER  OBJECTIVITY  AND  COMPLETENESS  WE  BREAK  DOWN  THE  AREAS 
LISTED  IN  DODI  5000-38  'PRODUCTION  READINESS  REVIEW'  AND  5000-34  'PRO¬ 
DUCTION  MANAGEMENT'  INTO  MORE  SPECIFIC,  DISCRETE  AREAS  AS  SHOWN  HERE-  WE 
HAVE  ALSO  DEVELOPED  OVER  700  CRITERIA  FOR  USE  BY  THE  PRR  TEAM  MEMBERS 
CONDUCTING  THE  REVIEW  SO  THAT  THEY  MAY  MORE  EFFECTIVELY  PERFORM  THE 
ON-SITE  REVIEW- 
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TYPICAL  TECHNICAL 
EXPERTISE  REQUIRED 


INDUSTRIAL  ENGINEERS 

MACHINIST 

AVIONICS 

SOFTWARE 

MATERIALS 

FINANCIAL 

SUPPLY 

PLANNING 

CONFIGURATION  CONTROL 

ILS 

SYSTEMS  INTEGRATION 

SIGNAL  SECURITY 

QUALITY  ASSURANCE 

PRODUCTION  CONTROL 

ELECTRICAL  ENGINEER 

AERO  ENGINEER 

METALURGIST 

CONTRACTS 

PROGRAM  ANALYST 

PROPULSION 

WEAPONS  MANAGEMENT 

TESTING 

BUDGET/FISCAL 

RELIABILITY 

THIS  IS  JUST  A  TYPICAL  LIST  OF  THE  VARIED  TYPES  OF  EXPERTISE  REQUIRED  AND 
IS  SOMETIMES  DIFFICULT  TO  HAVE  AVAILABLE  WHEN  THE  PRR  TEAM  SIZE  IS  SMALL 
AND  ALSO  WHEN  ONE  MUST  OPERATE  IN  A  MATRIX  ORGANIZATION  WHERE  THE  PRR 
TEAM  LEADER  MUST  SOMETIMES  'BEG*  FOR  A  CERTAIN  PERSON  TO  BE  A  MEMBER  OF 
THE  PRR  TEAM- 
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PRR  TEAM  MEMBER  SOURCES 


POP 


SERVICES 


product  engineering  services  office 

LOGISTICS  AGENCY 

CONTRACT  ADMINISTRATION  SERVICES 
CONTRACT  AUDIT  AGENCY 
SYSTEMS  MANAGEMENT  COLLEGE 


NAVAL  PRODUCT  ENGINEERING  SERVICES  OFFICE 
HQS  NAVAL  MATERIAL  COMMAND 
HQS  NAVAL  AIR  SYSTEMS  COMMAND 
NAVAL  AIR  TEST  CENTER  (PATUXENT  MD> 

NAVAL  WEAPONS  ENGINEERING  SUPPORT  ACTIVITY  (WASH  DC) 
NAVAL  AVIONICS  CENTER ( INDIANAPOLIS  IN) 

NAVAL  AIR  DEVELOPMENT  CENTER (WARMINSTER  PA) 

NAVAL  AIR  PROPULSION  CENTER  (TRENTON  NJ) 

NAVAL  WEAPONS  CENTER  (CHINA  LAKE  CA) 

NAVAL  WEAPONS  SUPPORT  CENTER  (CRANE  IN) 

NAVAL  PLANT  REPRESENTATIVE  OFFICE 


AIR  FORCE  PLANT  REPRESENTATIVE  OFFICE 


PRIVATE  INPUSTRY 


OTHER  GOVERNMENT 


UNIVERSITY  OF  PRINCETON 

vrnA  i mt 

FLIGHT  SYSTEMS,  INC. 
POTOMAC  RESEARCH,  INC. 
JVK  INTERNATIONAL 
TRW 

PLUS  OTHERS 


NATIONAL  SECURITY  AGENCY 

NATIONAL  AERONAUTICS  AND  SPACE  ADMINISTRATION 


OTHER  COUNTRY  GOVERNMENT  AGENCIES 


BRITISH  MINISTRY  OF  DCFCNCF 
CANADIAN  COMMERICAL  CORPORATION 


THIS  CHART  SHOWS  THE  MANY  ORGANIZATIONS  FROM  WHICH  WE  HAVE  OBTAINED  THE 
DESIRED  PRR  TEAM  MEMBERS- 


NOTE  THAT  IN  CANADA  AND  ENGLAND  WE  UTILIZED  PERSONNEL  FROM  THEIR  REVELANT 
ORGANIZATIONS- 
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PRR  TEAM  SIZE 


•  SYSTEM/SUBSYSTEM  CRITICALITY  OR 
COMPLEXITY 

•  CONTRACTOR  SIZE 

•  CONTRACTOR  ATTITUDE  OF  COOPERATION 

•  CONTRACTOR  TRACK  RECORD 

•  AVAILABILITY  OF  EXPERT  PERSONNEL 

•  FUNDING  AVAILABILITY 

•  DURATION  OF  ON-SITE  PRR 


THESE  ARE  THE  GENERAL  FACTORS  WHICH  AFFECT  THE  NUMBER  OF  PERSONNEL  ON  THE 
TEAM. 


HOWEVER,  OUR  EXPERIENCE  HAS  INDICATED  THAT  A  SHALL  TEAM  (12-20  PEOPLE)  OF 
QUALIFIED  EXPERTS  WHO  ARE  INDOCTRINATED  AND  KNOWLEDGEABLE  OF  THE  PURPOSE 
AND  PROCEDURES  OF  A  PRR  ALLOWS  FOR  CONDUCTING  AN  EFFICIENT  REVIEW  AND 
ELIMINATES  A  GREAT  DEAL  OF  IRRELEVANT  DISCUSSION  WHICH  IS  NORMALLY  ASSOCI¬ 
ATED  WITH  A  LARGE  GROUP- 


NOTE:  (SIMPLY  STATED  IT  ELIMINATES  A  LOT  OF  'BULL'  SESSIONS  AND 
THEREFORE  PROVIDES  GOOD  COMMUNICATION-) 


THE  MINIMIZATION  OF  TRAINEES/OBSERVERS/CURIOSITY  SEEKERS/NON-CONTRIBUTORS 
COULD  ENHANCE  THE  PROCESS-  HOWEVER,  POLITICAL  AND/OR  ORGANIZATIONAL 
FACTORS  SOMETIMES  HAKES  THIS  IMPRACTICAL-  NO  FORMAL  TRAINING  PROGRAM 
EXISTS  THEREFORE  WE  ARE  FORCED  TO  USE  ON  THE  JOB  TRAINING- 
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PRR  TEAM  ADMINISTRATIVE 
CHALLENGES 

•  AVAILABILITY  OF  REQUIRED  PERSONNEL 

•  INDOCTRINATION  OF  PRR  TEAM  MEMBERS 

•  FUNDING 

•  TRAVEL,  LODGING,  CLEARANCES 

•  LAST  MINUTE  CHANGES 


I  BELIEVE  THAT  ALL  OF  US  WHO  HAVE  BEEN  IN  KEY  ROLES  OF  THE  PRR  PROCESS 
RECOGNIZE  THE  PROBLEMS  AND  FRUSTRATIONS  OF  THE  CHALLANGES  LISTED  HERE. 


IT  SEEMS,  AND  MAYBE  BY  PURE  COINCIDENCE,  THAT  THE  PERSON  OR  PERSONS  YOU 
NEED  OR  DESIRE  ARE  NOT  AVAILABLE  FOR  MANY  STATED  REASONS- 


AS  PREVIOUSLY  SHOWN  INDOCTRINATION  OF  GOVERNMENT  TEAM  MEMBERS  IS  SEVERELY 
HAMPERED  BY  THE  FACT  THAT  TEAM  MEMBERS  ARE  COMING  FROM  VARIED  AND  WIDELY 
SCATTERED  GEOGRAPHICAL  LOCATIONS-  IN  AN  ATTEMPT  TO  OVERCOME  THIS,  WE  IN 
THE  NAVY  ASSURE  THAT  THERE  ARE  PRR  EXPERIENCED  MEMBERS  IN  EACH  FUNCTIONAL 
AREA  TO  BRIEF  THOSE  NON-EXPERIENCED  TEAM  PERSONNEL  THEIR  MEMBERS  AT  THE 
ON-SITE  LOCATION  PRIOR  TO  THE  REVIEW- 


LITTLE  NEED  BE  SAID  ABOUT  LAST  MINUTE  CHANGES,  THEY  ARE  A  'HEADACHE-' 
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GOVERNMENT  AND  INDUSTRY 
INDOCTRINATION  AND  COORDINATION 

•  JOINT  GOVERNMENT  AND  INDUSTRY 
INDOCTRINATION  MEETINGS  OF  KEY 
PERSONNEL 

•  EXCHANGE  OF  PRR  RELATED 
DOCUMENTATION 

»  COORDINATION  AND  NEGOTIATION  OF 
SCHEDULES,  PROCEDURES  AND  SUPPORT 
FOR  ON-SITE  REVIEW 


WE  HAVE  OBSERVED  THAT  THE  INDOCTRINATION  OF  CONTRACTORS,  ESPECIALLY  THOSE 
WHO  HAVE  NEVER  EXPERIENCED  A  PRR  TO  BE  A  CHALLENGING  EFFORT- 

THE  NAVY  INDOCTRINATES  CONTRACTORS  IN  ONE  OR  ALL  OF  THE  FOLLOWING  MANNERS: 

a)  VISIT  THEIR  FACILITY  AND  BRIEF  THEM  ON  THE  PRR  PROCESS 

b)  HAVE  CONTRACTORS  KEY  PERSONNEL  MEET  IN  WASHINGTON,  D-C- 

c)  PROVIDE  AND  EXPLAIN  THE  BASIC  POLICY  AND  GUIDES  SHOWN  EARLY  IN  THIS 
BRIEFING 

IN  SUMMARY  WE  MUST  LEAD  MANY  OF  THE  CONTRACTORS  BY  THE  HAND- 

VERY  INSTRUMENTAL  AND  KEY  TO  THIS  INDOCTRINATION  AND  COORDINATION  EFFORT 
IS  TO  HAVE  INDUSTRY  PROVIDE  ONE  AND  NOT  MORE  THAN  TWO  KEY  PERSONNEL  AS  A 
FOCAL  POINT  OR  POINT  OF  CONTACT  FOR  THE  PRR- 

THIS  PROVIDES  A  MOST  EFFICIENT  WAY  FOR  ACCOMPLISHING  THE  TASKS  IN 
BULLET  NUMBER  THREE - 
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PRR  DOCUMENTATION 


•  SELECT  APPROPRIATE  FUNCTIONAL  AREAS  FOR  REVIEW 

•  DEVELOP  RELEVANT  SPECIFIC  CRITERIA  FOR 
OBJECTIVITY  AND  COMPLETENESS 

•  ESTABLISH  AGENDA  FOR  PRR 

•  PROVIDE  TEAM  WORK  PACKAGES  WHICH  INCLUDE: 

•  INSTRUCTIONS 

•  CRITERIA 

•  DOCUMENTATION  FORMATS 

•  AGENDA 


THE  FINAL  STEP  IN  THE  PRR  PLANNING  PHASE  IS  THE  DEVELOPMENT  AND  OR 
AVAILABILITY  OF  REQUIRED  DOCUMENTATION  FOR  BOTH  THE  GOVERNMENT  PRR  TEAM 
AND  CONTRACTOR. 


THE  SIGNIFICANT  POINT  RELATED  TO  DOCUMENTATION  IS  IN  BULLET  NUMBER  FOUR. 
THIS  FUNCTION  SHOULD  BE  ACCOMPLISHED  AS  EARLY  AS  POSSIBLE  TO  ALLOW 
SUFFICIENT  PREPARATION  TIME  AND  FACILITATE  COORDINATION  BETWEEN  THE  PRR 
GOVERNMENT  TEAM  MEMBER  AND  HIS  COUNTERPART  IN  INDUSTRY. 
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IMPLEMENTATION  PHASE 


•  INITIAL  TEAM  MEMBER  BRIEFING 

•  BRIEFING  OF  PROGRAM  BY  LOCAL  GOVERNMENT  OFFICE 

•  BRIEFING  BY  PRIME  CONTRACTOR  AS  REQUIRED 

•  PRR  TEAM  LEADER  BRIEFING  TO  CONTRACTOR 

•  CONTRACTOR  BRIEFING  OF  PROGRAM 

•  CONDUCT  ON-SITE  PRR 

•  DOCUMENT  TEAM  LEADER  FINDINGS 

•  DAILY  REPORTING  TO  THE  TEAM  LEADER 

•  DAILY  CONTRACTOR  DEBRIEFING 

•  INFORMAL  FINDINGS  PROVIDED  TO  CONTRACTOR 


FOR  CLARIFICATION,  THE  IMPLEMENTATION  PHASE  ACTIVITIES  SHOWN  HERE  ARE 
THOSE  CONDUCTED  DURING  THE  PRR  AT  THE  CONTRACTOR'S  FACILITY. 

WE  HAVE  FOUND  ALL  THESE  ACTIVITIES  TO  BE  OF  BENEFIT  AND  IF  WE  WERE  TO 
LIST  THE  THREE  MOST  BENEFICIAL  TO  THE  NAVY  AND  THE  CONTRACTOR  THEY  WOULD 
INCLUDE  THE  LAST  THREE  BULLETS- 

THE  DAILY  REPORTING  TO  THE  TEAM  LEADER  BY  THE  TEAM  IMPROVES  THE  INTERACTION 
AND  EXCHANGE  OF  INFORMATION,  MINIMIZES  DUPLICATION  OF  GOVERNMENT  TEAM  AND 
INDUSTRY  EFFORT  PLUS  ASSURING  THAT  ALL  MEMBERS  OF  INDUSTRY  AND  GOVERNMENT 
ARE  'SINGING  THE  SAME  TUNE,*  THAT  MEANS  THEY  ARE  PROVIDING  THE  SAME 
FACTS- 

THE  DAILY  CONTRACTOR  DEBRIEFING  BY  THE  PRR  TEAM  LEADER  PRECLUDES  ANY 
SURPRISES,  MINIMIZES  MISUNDERSTANDINGS  OR  MIS-COHMUNICATION  AND  ASSISTS 
IN  RAPIDLY  RECTIFYING  PROBLEM  AREAS- 

PRIOR  TO  DEPARTING  THE  FACILITY,  THE  CONTRACTOR  IS  GIVEN  A  BRIEFING  OF 
THE  TOTAL  TEAM  FINDINGS  AND  PROVIDED  A  LIST  OF  POTENTIAL  PROBLEMS  FOR  HIS 
INFORMAL  USE-  THE  CONTRACTOR  IS  ALLOWED  TO  INVITE  ANY  OF  HIS  STAFF  TO 
THIS  BRIEFING  AND  WE  HAVE  FOUND  THAT  IT  IS  BENEFICIAL  FROM  A  PROFESSIONAL 
ASPECT  AND  GETTING  MANAGEMENT  ATTENTION- 
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COMPLETION  PHASE 


•  REPORT  FINDINGS  AND  RECOMMENDATIONS 
TO  PROJECT/PROGRAM  MANAGER 

•  ASSESS  RISK 

•  ALTERNATIVES  TO  REDUCE  OR  ELIMINATE 
RISK 

»  FORMAL  ACTION  REQUIRED  OF  CONTRACTOR 

•  FOLLOW-UP  TO  COMPLETION 

•  DOCUMENT  FINDINGS  FOR  DSARC 


THE  COUPLET  ION  PHASE  IS  PRIMARILY  THE  RESPONSIBILITY  OF  THE  NAVY  PROJECT/ 
PROGRAM  MANAGER.  IT  IS  ONLY  HE  WHO  HAS  THE  CHARTER  AND  RESPONSIBILITY  TO 
EFFECTIVELY  MANAGE  HIS  WEAPON  SYSTEM- 


FROM  THE  NAVWPNENGSUPPACT  PERSPECTIVE  THE  TWO  MOST  IMPORTANT  ASPECTS  OF 
THIS  PHASE  ARE  TO  ENSURE  THAT  THE  ACTION  ITEMS  ARE  TRACKED  AND  THE  STATUS 
OF  THEIR  PROGRESS  TO  COMPLETION  IS  MONITORED  PLUS  THE  ESTABLISHMENT  OF  A 
REPOSITORY  OF  DATA  SO  THAT  THE  FINDINGS  CAN  BE  DOCUMENTED  FOR  USE  AT 
DSARC. 


THE  FOLLOW  UP  TO  COMPLETION  COULD  BECOME  AN  EXTENDED  TASK  IF  THE  DATES 
ESTABLISHED  TO  COMPLETE  THE  CORRECTIVE  ACTIONS  FOR  SOLVING  THE  IDENTIFIED 
DEFICIENCIES  ARE  NOT  REALISTIC. 
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PROGRAM/PROJECT  MANAGEMENT 
RESPONSIBILITIES 


•  CONTRACTURAL  REQUIREMENT  FOR  PRR 

•  FUNDING  FOR  CONTRACTOR  AND  GOVERNMENT 
AGENCIES 

•  COORDINATE  PRR  SCHEDULE 

•  ESTABLISH  REALISTIC  LEAD  TIMES  PRIOR  TO  DSARC 

•  EVALUATE,  COORDINATE  AND  MONITOR  PROGRESS 
OF  PRR  TEAM  ACTION  ITEMS 

•  FOLLOW-UP  WITH  SUBSEQUENT  REVIEWS  TO  TRACK 
PROGRESS 

•  OOCUMENTS  FINOINGS  IN  A  OATA  REPOSITORY 

•  RESOLVE  ISSUES  FOR  ADDRESSAL  AT  DSARC 


THIS  LISTING  OF  RESPONSIBILITIES  ENCOHPASS  ACTIVITIES  MHICH  RUST  BE 
ACCOMPLISHED  FROM  THE  POINT  IN  TIRE  THAT  THE  DECISION  WAS  MADE  THAT  A 
PROGRAM  WILL  HAVE  A  PRR  UP  TO  THE  DSARC  MILESTONE- 


OF  THESE  RESPONSIBILITIES,  THE  LAST  ONE  IS  PROBABLY  THE  HOST  IMPORTANT  TO 
THE  OVERALL  PROGRAM  AND  THIS  REQUIRES  CLOSE  COORDINATION  AND  TECHNICAL 
EFFORT  AMONG  THE  PARTICIPANTS  WHICH  INCLUDE  PROGRAM  MANAGERS,  DPESO, 
NPESO,  PRIME  CONTRACTOR/SUBCONTRACTOR,  NSARC  AND  DSARC  PRINCIPALS- 
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PRR  ACTIVITY  CHART 


OFFICIAL  I0TIFICATI0I 
OF  ACTIOI.  ITEMS  CLOSEO 


/ 


M 

IEVIEV  OF 

COITIACTOI  COMPETED 
ACTION 


\ 


ACTIOI  IT  COITIACTOI 
TONIIINIZE  POTEITIAL 
PI 


-SITE 
IIITIAL  IIIEF 


41  -  SITE 

Pit 


completion 

PHASE 


IMPLEMENTATION 

PHASE 


01- SI  IE 
PUT  IIIIF 


EVALUATION  OF 
POTEITIAL  PIOILENS 


OFFICIAL  TIAISNITTAL 
OF  ACTIOI  ITEMS 
TO  COITIACTOI) 


NOTIFICATIOI  TO 
COITIACTOI  OF 
FUICTIOIAL  AIEAS  / 
FACTOIS /  CIITEIIA 
FOI  01 -SITE  Pll 


PLAN 


PE trill 


IN  A  SUMMARY  MANNER  THIS  SHOWS  THE  ACTIVITIES  AND  PRINCIPLE  PHASES  OF  THE 
PRR  PROCESS  WHICH  WERE  JUST  DISCUSSED-  THE  ACTIVITIES  BEGIN  AT  THE  TOP  AND 
PROCEED  CLOCKWISE. 


THERE  ARE  THREE  ADDITIONAL  CHARTS  WHICH  LIST  THE  DESIRED  BENEFITS  OF  THE 
PRR  PROCESS;  THE  DYNAMICS  OF  THE  PRR  AND  A  SUMMARY  OF  THE  MORE  SIGNIFICANT 
PRR  LESSONS  LEARNED-  I  WILL  VERY  BRIEFLY  DISCUSS  EACH  OF  THESE- 
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PRR  DESIRED  BENEFITS 


•  TIMELY  IDENTIFICATION  AND  RESOLUTION  OF 
POTENTIAL  PROBLEMS 

•  COST  SAVINGS  AND  COST  AVOIDANCE 

•  INCREASED  CONFIDENCE  IN  ACHIEVEMENT  OF 
PROGRAM  OBJECTIVES 

•  GREATER  ASSURANCE  OF  ORDERLY 
TRANSITION  TO  PRODUCTION 

•  DSARC  APPROVAL 


THE  HOST  DESIRED  BENEFIT  TO  BE  DERIVED  FROM  THE  PRR  PROCESS  IS  TO 
ASSURE  THAT  THE  PROGRAM  RISKS  HAVE  BEEN  REDUCED  TO  AN  ACCEPTABLE  LEVEL 
WHICH  RESULTS  IN  DSARC  APPROVAL  TO  PROCEED  INTO  PRODUCTION- 
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DYNAMICS  OF  PRR 
IMPLEMENTATION  PROCESS 

•  CONTINUOUS  SCHEDULE  CHANGES 

•  PRR  SIZE  AND  TEAM  MEMBER  CHANGES 

•  REVISIONS  TO  PRR  PROCEDURES 

•  NEW  AND  VARIED  ATTITUDES  OF  CONTRACTORS/SUBCONTRACTORS 

•  AVAILABILITY  OF  TIME,  MANPOWER  AND  MONETARY  RESOURCES 

•  EXPERTISE  AND  OBJECTIVITY  OF  PRR  TEAM  MEMBERS 

•  VARIED  DOD  AND  SERVICE  AREAS  OF  EMPHASIS 

•  NAVY  MANAGEMENT  ABILITY  TO  ENSURE  CONTRACTOR  ACTION 
FOR  PROBLEM  RESOLUTION 


FROM  OUR  EXPERIENCE,  THESE  ARE  THE  MAJOR  ELEMENTS  THAT  CAUSE  THE  PRR 
PROCESS  TO  BE  ONE  OF  A  DYNAMIC  NATURE. 


(PAUSE  AND  LET  THEM  LOOK  AT  THE  SLIDE) 


ASSOCIATED  WITH  ANY  DYNAMIC  PROCESS  THERE  IS  AN  INHERENT  NEED 

FOR  RESPONSE  IN  THE  FORM  OF  'LESSONS  LEARNED-'  OUR  LESSONS  LEARNED  ARE 

SHOWN  ON  THE  NEXT  AND  FINAL  SLIDE • 
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SUMMARY  OF  PRR  LESSONS  LEARNED 


•  EARLY  PLANNING  ANO  COORDINATION  IS  VITAL 

•  FLEXIBILITY  FOR  SCHEDULE  ANO  MANPOWER  ALLOCATION  IS  REQUIREO 

•  INDOCTRINATION  OF  CONTRACTOR  AND  GOVERNMENT  REQUIRED 

•  FORMALIZATION  OF  TRAINING  VERY  OESIRABLE 

•  FUNDING  SHOULD  BE  PROVIOEO  EARLY 

•  CORRECTIVE  ACTION  TIME  FOR  POTENTIAL  PROBLEMS  SHOULO  BE  REOUCEO* 

•  PROCEDURES  FOR  TRANSFER  OF  KNOWLEDGE  AMONG  SERVICES  OF  CONTRACTOR’S 
PERFORMANCE  SHOULO  BE  EXPLORED 

•  REDUCTION  OF  DUPLICATIVE  GOVERNMENT  EFFORTS  SHOULO  BE  EXAMINEO 

•  THE  DEVELOPMENT  OF  MORE  FINITE  METHOOS  FOR  PRR  RISK  ASSESSMENTS 
SHOULO  BE  STUOIEO 

•  ALL  FUNCTIONAL  AREAS  SHOULO  BE  REVIEW EO  SIMULTANEOUSLY 

•  OAILY  FUNCTIONAL  AREA  OEBRIEF  IS  VERY  EFFECTIVE 

•  PRESENCE  OF  NAVY  PRR  TEAM  STIMULATED  CONTRACTOR 


THIS  SUMMARY  LIST  OF  LESSONS  LEARNED  IN  NOT  RANKED  IN  ANY  ORDER  OF 
IMPORTANCE.  FROM  A  PERSONAL  VIEWPOINT,  I  CONSIDER  THAT  THE  FORMALIZATION 
OF  TRAINING  RELATED  TO  THE  PRR  PROCESS  IS  NECESSARY.  WE  MUST  ASSURE  THAT 
THERE  IS  A  READY  POOL  OF  EXPERIENCED  TALENT  AVAILABLE  SO  THAT  THE  PERSONAL 
BURDENS  OF  THE  PRESENTLY  EXISTING  SHALL  NUMBER  OF  EXPERTS  CAN  BE  REDUCED- 
THE  PRR  EFFORT  IS  A  VALUABLE  AND  IMPORTANT  ONE  AND  ACCOMPLISHING  THE  MANY 
ASSOCIATES  TASKS  SHOULD  NOT  BE  THE  RESPONSIBILITY  OF  A  FEW  GOOD  PEOPLE- 


THIS  CONCLUDES  HY  BRIEFING 
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F/A-18 


PRODUCTION  READINESS  REVIEW 
PROGRAM 


CAPTAIN  L.B.  GALL  I  ON,  USN,  (RET) 


F/A-18  PRR  PROGRAM 

•  OVERVIEW 

•  SCHEDULE 

•  PLANNING 

•  KEY  FEATURES 

•  LESSONS  LEARNED 

•  RECOMMENDATIONS 


141 


OVERVIEW 


0  DEFINITION 
t  POLICY  DIRECTIVES 
0  APPROACH 
0  IMPLEMENTATION 


OVERVIEW 


0  DEFINITION: 

0  A  FORMAL  REVIEW,  SUPPORTED  BY  A  SERIES  OF  INCREMENTAL  EVALUATIONS  TO  DETERMINE: 

0  WHETHER  A  SYSTEM  OR  EQUIPMENT  UNDER  DEVELOPMENT  IS  READY  FOR  EFFICIENT 
AND  ECONOMICAL  QUANTITY  PRODUCTION. 

0  IF  ADEQUATE  PLANNING  FOR  THE  PRODUCTION  PHASE  HAS  BEEN  ACCOMPLISHED. 

0  WHETHER  MANUFACTURING/ENGINEERING  PROBLEMS  ENCOUNTERED  DURING  DEVELOPMENT 
HAVE  BEEN  RESOLVED. 
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OVERVIEW 


•  POLICY  DIRECTIVES  -  1977 

•  DOD  DIRECTIVE  5000.1  -  18  JAN  1977 

•  DOD  DIRECTIVE  5000.2  -  18  JAN  1977 

•  DECISION  COORDINATING  PAPER 

•  NEW/REVISED  POLICY  DIRECTIVES  -  1980 

•  DOD  DIRECTIVE  5000.1  -  19  MAR  1980 

•  DOD  DIRECTIVE  5000.2  -  19  MAR  1980 

t  DOD  INSTRUCTION  5000.38  -  2A  JAN  1979 


OVERVIEW 


•  APPROACH 

•  REVIEWED  OTHER  SERVICES'  POLICY 
DIRECTIVES  AND  PROCEDURES 

•  CONTACTED  VENDORS  WHO  HAD 
COMPLETED  REVIEWS 

•  SOLICITED  COMMENTS  FROM 

DOD  PRODUCTION  MANAGEMENT  PERSONNEL 
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OVERVIEW 


•  IMPLEMENTATION 

•  DEVELOPED  F/A-18  PRODUCTION 
READINESS  REVIEW  PLAN 

•  REQUESTED  REVIEW  BY  OSD,  NAVMAT, 

NAVY  AND  CONTRACTOR  PERSONNEL 

•  PLAN  APPROVED  BY  CHIEF  OF  NAVY  MATERIAL 
MARCH  1978  AND  PROMULGATED 


SCHEDULE 

•  FIRST  INCREMENTAL  PRRs  FOR  PROGRAM  REVIEW  NOVEMBER  1978 

•  JUNE  1978  -  AUGUST  1978 

•  5-9  JUNE  78 . MCAIR 

•  19-22  JUNE  78 . NORTHROP 

•  14-17  AUGUST  78  .  .  .  .GENERAL  ELECTRIC 

•  29-31  AUGUST  78  .  .  .  .HUGHES 

•  SELECTED  SUBCONTRACTOR  PRRs 

•  DECEMBER  1978  -  NOVEMBER  1979 

•  SECOND  INCREMENTAL  PRR 

•  FEBRUARY  1979  -  APRIL  1979 

•  THIRD  INCREMENTAL  PRR  FOR  DSARC/PROGRAM  REVIEW  MARCH  1980 

•  OCTOBER  1979  -  DECEMBER  1979 

•  FOURTH  INCREMENTAL  PRR  FOR  DSARC  OCTOBER  1980 

•  JUNE  1980  -  AUGUST  1980 
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PLANNING 


•  FUNCTIONAL  AREAS 

•  REVISED  FUNCTIONAL  AREAS 

•  SAMPLE  TEAM  COMPOSITION 

•  SAMPLE  DAILY  SCHEDULE 

•  SUMMARY  REPORT  FORMAT 


PLANNING 


•  FUNCTIONAL  AREAS 

•  PROGRAM  MANAGEMENT 

•  ENGINEERING  PRODUCT/DESIGN 

•  PRODUCTION  COST 

•  MAKE  OR  BUY 

•  TESTING  STATUS 

•  SUBCONTRACTING 

•  MANUFACTURING  PLANNING 

•  SOFTWARE 


•  PRODUCTION  METHODS  AND 
PROCESSED 

•  PRODUCTION  CONTROL 

•  FACILITIES  AND  EQUIPMENT 

•  TOOLING  AND  TEST  EQUIPMENT 

•  MATERIAL 

•  MANPOWER 

•  PLANT  LAYOUT 

•  QUALITY  ASSURANCE 

•  INTEGRATED  LOGISTICS  SUPPORT 
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ELMMlMfi 


•  REVISED  FUNCTIONAL  AREAS 

•  PROGRAM  MANAGEMENT 

•  PRODUCTION  STATUS 

•  DEVELOPMENT 

•  QUALITY  ASSURANCE 

•  INTEGRATED  LOGISTICS  SUPPORT 

•  SOFTWARE  MANAGEMENT 


PLANNING 


SAMPLE  TEAM  COMPOSITION 
GENERAL  ELECTRIC  PRODUCTION  READINESS  REVIEW 
(F404  ENGINE) 

TEAM  LEADER  GROUP 


CAPT  L.B>  GALL  I  ON,  PMA265-5 
MR,  R,  STEPLER,  AIR5123F2 
MR.  S,  PATELLOS,  FSI 


TEAM  LEADER 


ASSISTANT  TEAM  LEADER 


ADVISOR  TO  TEAM  LEADER 


PROGRAM  MANAGEMENT 


FUNCTIONAL  AREA  GROUPS 
INTEGRATED  LOGISTICS  SUPPORT 


QUALITY  ASSURANCE 


CDR  W,  KRIEG,  PMA-265-2  *MR ,  J.  SYLVESTER,  AIR4105A 


MR.  J.  MCBRIDE,  ESA-672B 


PRODUCTION  STATUS 


MR.  N.  GALLIPOLI,  AIR4113A5 


OBSERVERS 

MR.  SULLIVAN,  DPESO 
MR.  W.  OAKS,  NPESO 


CDR  W,  BROWN,  PMA265-B  LCDR  R.  KIEFER,  PMA265-C 

MR,  E.  CARROLL,  ESA-4B  DEVELOPMENT  STATUS 

MR.  E,  SHINN,  ESA-61  *MR ,  R,  PRINE,  AIR53613A 


MR.  J.  WHITT,  DPESO 
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PLANNING 


SAMPLE  DAILY  SCHEDULES: 

FIRST  DAY  SCHEDULE 
0800  NAVY  TEAM  MEETING 
0900  BRIEF  BY  LOCAL  GOVERNMENT 
REPRESENTATIVE 

1000  BRIEF  BY  PRIME  CONTRACTOR 
TO  NAVY  TEAM 

1100  BRIEF  TO  CONTRACTOR  BY  PRR 
TEAM  LEADER 
1200-1300  LUNCH 

1300  BRIEF  BY  CONTRACTOR  TO  NAVY 
TEAM  AND  PRIME  CONTRACTOR 
1430  ONE  ON  ONE  DISCUSSIONS 
1600  MEETING  WITH  PRR  TEAM  LEADER 
1700  DEBRIEF  CONTRACTOR  ON 
SIGNIFICANT  EVENTS 


PLANNING 

SUMMARY  REPORT  FORMAT* 

•  NUMBER  OF  CRITERIA  EVALUATED . 

•  NUMBER  OF  CRITERIA  FOUND  SATISFACTORY  . 

•  NUMBER  OF  EXCEPTIONAL  FINDINGS:  . 

•  NUMBER  OF  POTENTIAL  PROBLEMS:  . 

•  IN  WORK . 

9  OPEN  i  i  i  i  i  i  i  ■  i  i  i  i  i  i  i  i  i  i  i  ■  ■  i  ■  i  i  ■ 

•  TOTAL  POTENTIAL  PROBLEMS:  . 

•  INITIAL  RISK  ASSESSMENT . 

0  HIGH . . . 

•  MEDIUM  . 

®  LOW 

•  NO  KNOWN  . 

*  Prepared  for  each  functional  area  reviewed 


LAST  DAY  SCHEDULE 
0800-1000  CONTINUE  ONE  ON  ONE 
1000-1100  PREPARE  ACTION  ITEMS  AND 
COORDINATE  WITH  PRIME  AND 
SUBCONTRACTOR 

1100-1200  PREPARE  EXIT  DEBRIEF 
1300-  EXIT  DEBRIEF  FOR  CONTRACTOR 
BY  NAVY  TEAM  MEMBERS 
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KEY  FEATURES 


•  PROJECT  OFFICE  DESIGNATED  CENTRAL  POINT  FOR 
OVERALL  MANAGEMENT 

•  FACTORS  AND  CRITERIA  DEVELOPED  FOR  EACH 
FUNCTIONAL  AREA 

•  ALL  FUNCTIONAL  AREAS  REVIEWED  AT  SAME  TIME 

•  TEAM  SIZE,  SELECTION  AND  TRAINING  CLOSELY 
MONITORED 

•  CONTRACTOR  PRE  BRIEFS  AND  EXIT  DEBRIEFS 
DEVELOPED 

•  DETAILED  SCHEDULE  COORDINATED  WITH  CONTRACTOR 

•  INITIAL  SUB-CONTRACTOR  SELECTION  COMPLETED 

f  ACTION  ITEM  REPORTING  PROCEDURES  IMPLEMENTED 


LESSONS  LEARNED 


•  REQUISITE  PLANNING 

•  DEDICATED  PROJECT  OFFICE  COORDINATION 

•  FLEXIBLE  REVIEW  PROCESS 

•  EARLY  OSD  ROLE  DEFINITION 

•  P0SSI3LE  JOINT  SERVICE  APPROACH 

•  TIMELY  ACTION  ITEM  FOLLOW  UP  SYSTEM 

•  CONSISTENT  TEAM  MEMBERSHIP 
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RFCnHHENDATIONS 


DON'T  INSTITUTIONALIZE  THE  PROCESS 
PROJECT  OFFICE  HAS  KEY  ROLE  IN  THE  PROCESS 
FLEXIBILITY  MUST  BE  MAINTAINED 

•  SCHEDULES 

•  STATUS  OF  PROGRAM 

TIMELY  IMPLEMENTATION  WILL  PROVIDE 
DESIRED  RESULTS 
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TITLE:  A  PRAGMATIC  LOOK  AT  THE  PRODUCTION  READINESS  REVIEW  PROCESS  FROM 
THE  NAVPRO  PROSPECTIVE 

AUTHOR:  H.  A.  MCCORMICK,  DIRECTOR  OF  THE  INDUSTRIAL  DIVISION,  NAVPRO  LYNN 
AUTHOR'S  BACKGROUND 

THE  AUTHOR  RECEIVED  A  BSBA  IN  1959  FROM  BOSTON  COLLEGE,  MAJORING  IN 
INDUSTRIAL  MANAGEMENT. 

HE  IS  A  1974  GRADUATE  OF  THE  CORNELL  BUSINESS  AND  PUBLIC  ADMINISTRATION 
GRADUATE  SCHOOLS  EDUCATION  FOR  PUBLIC  MANAGEMENT  PROGRAM. 

MR.  MCCORMICK  PREVIOUSLY  GAVE  A  RESEARCH  PAPER  AT  THE  DOD'S  SIXTH 
ANNUAL  PROCUREMENT  RESEARCH  SYMPOSIUM  HELD  AT  THE  U.S.  MILITARY  ACADEMY, 

WEST  POINT,  NEW  YORK  IN  1977.  THE  PAPER  WAS  ENTITLED  "WORKLOAD  ASSESSMENT 
AND  MANPOWER  APPORTIONMENT  IN  DCASR  BOSTON'S  PRODUCTION  DIRECTORATE." 

HE  HAS  HAD  SEVERAL  MANAGEMENT  ASSIGNMENTS  IN  PLANT  OFFICES  AND  AS  A 
REGIONAL  STAFF  MANAGER,  DURING  WHICH  HE  PARTICIPATED  IN  MANY  PRODUCTION 
READINESS  REVIEWS. 

DURING  HIS  TENURE  WITH  DCAS,  HE  HAS  HAD  SPECIAL  ASSIGNMENTS  FOR  DLA 
HEADQUARTERS. 

HE  CURRENTLY  HAS  A  SPECIAL  ASSIGNMENT  FROM  NAVAIR  HEADQUARTERS  (AIR- 
519)  TO  IMPLEMENT  AND  TEST  THE  NEW  SYSTEM  CONCEPT  FOR  NAVY  PLANT  OFFICES 
CALLED  CONTRACTOR  OPERATING  SYSTEM  EVALUATION  AND  MONITORING  PROGRAM  OR 
COSEMP. 
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•  NOW  THAT  THE  POLITICAL  SEASON  IS  OVER,  THE  POLITICIANS  ARE  FAIR  GAME. 

•  AT  A  HOT  CHICAGO  RALLY  WILLIAM  JENNINGS  BRYAN  WAS  HOLDING  FORTH  IN  HIS 
BEST  ORATORICAL  STYLE. 

"WHILE  WE  ARE  MEETING  HERE,"  HE  SAID,  "MRS.  BRYAN  IS  SLEEPING  IN  A 
HUMBLE  BOARDINGHOUSE  DOWN  BY  THE  STOCKYARDS.  BUT  THAT'S  ALL  RIGHT,  COME  NEXT 
MARCH  SHE'LL  BE  SLEEPING  IN  THE  WHITE  HOUSE." 

THAT  WAS  TOO  MUCH  FOR  A  HECKLER  IN  THE  CROWD,  WHO  PROMPTLY  PIPED,  "IF 
SHE’S  IN  THE  WHITE  HOUSE  IN  MARCH  SHE'LL  BE  SLEEPING  WITH  MCKINLEY." 

SLIDE  §]  ON 

•  NOW  THAT'S  CRITICISM.  AS  PARTICIPANTS  IN  PRODUCTION  READINESS  REVIEWS, 
WE  ARE  CRITICS. 

•  HENRY  JAMES  SAYS  OF  CRITICISM: 

"THE  CRITICAL  SENSE  IS  SO  FAR  FROM  FREQUENT  THAT  IT  IS  ABSOLUTELY  RARE, 
AND  THE  POSSESSION  OF  THE  CLUSTER  OF  QUALITIES  THAT  MINISTER  TO  IT  IS  ONE  OF 
THE  HIGHEST  DISTINCTIONS ....  IN  THIS  LIGHT  ONE  SEES  THE  CRITIC  AS  THE  REAL 
HELPER  OF  THE  ARTIST,  A  TORCHBEARING  OUTRIDER,  THE  INTERPRETER,  THE  BROTHER.. 
JUST  IN  PROPORTION  AS  HE  IS  SENTIENT  AND  RESTLESS,  JUST  IN  PROPORTION  AS  HE 
REACTS  AND  RECIPROCATES  AND  PENETRATES  IS  THE  CRITIC  A  VALUABLE  INSTRUMENT." 
SLIDE  | 1  OFF 

•  SO  MUST  WE  IN  THE  PRR  BE  A  VALUABLE  INSTRUMENT  AND  PENETRATE  AS 
CONSTRUCTIVE  CRITICS. 

•  CAPTAIN  COLE,  USN ,  HAS  CHARACTERIZED  THE  PRR  IN  THE  JULY-AUGUST  ISSUE  OF 
THE  PROGRAM  MANAGER  AS  FOLLOWS: 

"THE  ESSENCE  OF  THE  PRODUCTION  READINESS  REVIEW  IS  A  ONE-ON-ONE  CONSULTA 
TION  BETWEEN  THE  MILITARY  DEPARTMENT  AS  THE  CUSTOMER  AND  SYSTEM  OR  SUBSYSTEM 
CONTRACTOR." 
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SLIDE  n  ON 


•  THE  PURPOSE  OF  THE  REVIEW  IS  TO  ASSESS  THE  PRODUCIBILITY  OF  THE  SYSTEM 
AT  THE  MANUFACTURING  FACILITY. 

•  BY  MAKING  A  CRITICAL  REVIEW  OF  CURRENT  STATUS,  FUTURE  RISKS  WILL  BE 
MINIMIZED  AND  PROBLEM  AREAS  WILL  SURFACE  EARLY. 

•  THIS  (PRR)  IS  MANAGEMENT  BY  OBJECTIVE  IN  ITS  CLASSICAL  SENSE.... TO  USE 

A  HACKNEYED  PHRASE _ THIS  IS  WHERE  THE  RUBBER  MEETS  THE  ROAD. 

•  WE  AT  THE  PLANTS  SHOULD  BE  OPTIMISTIC  WITH  A  HEALTHY  DEGREE  OF  SKEPTI¬ 
CISM. 

SLIDE  #2  OFF 

•  THE  PRR  IS  PRIMARILY  MANUFACTURING  ORIENTATED,  BUT  IT  TRANSCENDS  MISSIONS 
AND  DEPARTMENTS. 

r 

SLIDE  #3  ON 

•  THERE  IS  NO  SUBSTITUTE  FOR  DETAILED  PLANNING. 

•  WE  NOT  ONLY  WANT  A  ROAD  MAP,  WE  WANT  THE  TRAFFIC  LIGHTS,  BRIDGES,  RAIL¬ 
ROAD  CROSSINGS,  DETOURS,  AND  THE  WEATHER  FORECAST. 

•  SIGNIFICANT  REVISIONS  TO  DOD  DIRECTIVES  5000.1  AND  5000.2  WERE  SIGNED  BY 
THE  SECRETARY  OF  DEFENSE  W.  GRAHAM  CLAYTOR,  JR.  ON  19  MARCH  1980. 

•  THE  INTENT  OF  THE  REVISIONS  IS  TO  STREAMLINE  THE  ACQUISITION  PROCESS  FOR 
PROGRAM  MANAGERS  AND  PINPOINT  RESPONSIBILITIES. 

SLIDE  #3  OFF 

•  LET'S  LOOK  AT  A  SUMMARY  OF  THE  REVISIONS  AND  THE  LINKAGE  TO  THE  IMPORTANT 
ROLE  THAT  THE  PLANT  OFFICE  PERFORMS  IN  THIS  PROCESS,  BY  THE  FLOWDOWN  OF  POLICY 
CHANGES. 

SLIDE  #4  ON 

•  THERE  ARE  FOUR  NEW  POLICY  STATEMENTS  IN  DOD  DIRECTIVE  5000.1:  AFFORD¬ 
ABILITY,  ACQUISITION  TIME,  STANDARDIZATION,  AND  LOGISTIC  SUPPORTABILITY . 
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•  FIRST,  THE  AFFORDABILITY  ASPECT .... DOD*  1 S  CONSERVATIVE  ESTIMATES  INDICATE 
THAT  ANTICIPATED  PROCUREMENT  REQUIREMENTS  OVER  THE  NEXT  15  YEARS  WILL  EXCEED 
ANTICIPATED  APPROPRIATIONS  BY  A  FACTOR  OF  TWO. 

•  SECOND,  THE  ACQUISITION  TIME.... IS  A  RECOGNITION  THAT  TIME  LOST  EQUATES 

TO  EXCESS  COSTS  FROM  BOTH  INFLATIONARY  AND  OBSOLESCENT  VIEWPOINTS.  A  COROLLARY 
TO  THESE  FACTORS  IS  THE  LONGER  THE  ACQUISITION  CYCLE  THE  SHORTER  THE  FLEET  USE 
DUE  TO  TECHNOLOGICAL  ADVANCEMENTS. 

I  NEXT,  STANDARDIZATION:  FORD  MOTOR  COMPANY  HAS  THEIR  WORLD  CAR.... WE  FOR 

OUR  OWN  AND  ALLIED  COUNTRIES  REQUIRE  STANDARDIZATION  FOR  INTERCHANGEABILITY 
AND  PRODUCIBILITY  AS  WELL  AS  MAINTAINABILITY  AND  SUPPORTABILITY . 

I  THE  FOURTH  POLICY  STATEMENT,  LOGISTIC  SUPPORTABILITY,  SHARES  EQUAL 

SALIENCY  WITH  THE  OTHER  THREE.  THE  ADEQUACY  OF  LOGISTICS  PLANNING  WILL  NOW 
BE  CONSIDERED  EARLIER  AT  MILESTONE  II,  RATHER  THAN  MILESTONE  III,  TO  WHICH 
THE  LOGISTICIANS  SAY  AMEN . 

SLIDE  #4  OFF 

•  PRIOR  TO  DISCUSSING  THE  PLANT  OFFICE  PRR  ROLE,  LET'S  LOOK  AT  THE 
DISCUSSION  ITEMS  WHICH  ARE  NEW  IN  THE  INITIAL  PLANNING  STATEMENT  (IPS)  AND 
ARE  IN  FACT  GERMANE  TO  THE  PLANT  OFFICE. 

SLIDE  45  ON 

•  COST  EFFECTIVENESS  ANALYSIS 

I  ORGANIZATIONAL/OPERATIONAL  CONCEPT 

•  DATA  MANAGEMENT 

•  CONFIGURATION  MANAGEMENT 

•  COST 

PRODUCTION  -  SECOND  SOURCE 
SOURCE  AND  APPLICATION  OF  FUNDS 
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SLIDE  #5  OFF 


SLIDE  ii 6  ON 

*  RELIABILITY  AND  MAINTAINABILITY 

0  QUALITY 

I  TRAINING 

0  FACILITIES 

0  COMPUTER  RESOURCES 

•  ENERGY 

0  HEALTH  AND  SAFETY 

SLIDE  16  OFF 

0  THE  PLANT  OFFICE  SHOULD  ACT  AS  COORDINATOR  BOTH  INTERNALLY  AND 

EXTERNALLY. 

SLIDE  #7  ON 

0  INTERNAL  COORDINATION  WITHIN  THE  INDUSTRIAL  DIVISION  (MANUFACTURING) 

AND  ACROSS  FUNCTIONS  WITH  QUALITY,  CONTRACTS  AND  ENGINEERING  DIVISIONS  TO 
ASSURE  THE  APPROPRIATE  SKILLS  ARE  MOBILIZED  FOR  THE  REVIEW  TEAM. 

0  EXTERNAL  COORDINATION  CONSISTS  OF  THE  CAO  TEAM  INTERFACING  WITH  THE 

CONTRACTOR'S  MANAGEMENT  FOR  CONSOLIDATED  SYSTEMS  UPDATE  ORIENTATION  AND 
FUTURE  PRR  SUB-TEAM  TASK  FORCE  ASSIGNMENTS. 

0  EXTERNAL  COORDINATION  REQUIRES  PLANNING  AND  INTEGRATION  OF  THE  PRODUCTION 

READINESS  REVIEW  TEAM  WITH  THE  PLANT  OFFICE  REPRESENTATIVES  AND  CONTRACTOR 
MANAGEMENT. 

0  THE  IMPORTANCE  OF  THESE  COORDINATION  EFFORTS  CANNOT  BE  OVEREMPHASIZED 

IN  ORDER  TO  MAKE  THE  GREATEST  USE  OF  THE  LIMITED  TIME  THE  PRR  TEAM  HAS  AT  A 
GIVEN  LOCATION. 

SLIDE  #7  OFF 


155 


SLIDE  #8  ON 


I  THE  INITIAL  MEETING  (GOVERNMENT,  NAVPRO,  AND  PRR  TEAM)  MUST  BE 

STRUCTURED  IN  AN  EFFICIENT  MANNER  FOR  AN  EXECUTIVE  SUMMARY  OF  SIGNIFICANT 
EVENTS.... PARTICULARLY  PROBLEM  AREAS  IN  ORDER  THAT  THE  TEAM  CAN  ZERO  IN  ON 
THESE  AREAS  EARLY  AND  SURFACE  THEM  FOR  RESOLUTION. 

I  A  SINGLE  SPOKESPERSON  WITH  NAVPRO  TEAM  BACK-UP  FOR  CLARIFICATION  OF 

SPECIFIC  AREAS  OF  TECHNICAL  EXPERTISE  IS  HIGHLY  EFFECTIVE. 

•  THE  NAVPRO  TEAM  CAN  BE  INVALUABLE  TO  THE  REVIEW  EFFORT  BY  ITS  INTENSIVE 
KNOWLEDGE  OF  THE  CONTRACTOR'S  MANUFACTURING  CAPABILITY. 

•  AS  OUR  FRIEND  ON  MONDAY  NIGHT  FOOTBALL  SAYS:  "TELL  IT  LIKE  IT  IS." 

•  I  LIKEN  THE  PLANT  REPRESENTATIVE'S  ROLE  TO  THAT  OF  THE  SYMPHONIC 

ORCHESTRA  CONDUCTOR - NOT  ONLY  SHOULD  THE  PLANT  REPRESENTATIVE  HAVE  THE 

BROAD  OVERVIEW  OF  THE  CONTRACTOR'S  CAPABILITIES  AND  POTENTIAL  PITFALLS.... 
BUT  ALSO  HAVE  THE  ORCHESTRA  LEADER'S  DETAILED  KNOWLEDGE  OF  THE  INDIVIDUAL 
AND  COLLECTIVE  INSTRUMENTS  TO  DIRECT  A  HARMONIOUS  SYMPHONY  THAT  FOLLOWS  THE 
SCORE  OR  PLAN  FOR  A  SUCCESSFUL  EXECUTION  TO  RECEIVE  THE  PLAUDITS  OF  THE 
CUSTOMERS. 

SLIDE  #8  OFF 

•  LET'S  REVIEW  SOME  OF  THE  MORE  IMPORTANT  ELEMENTS  THAT  THE  FIELD 
ACTIVITY  MUST  BE  ON  TOP  OF  FOR  A  SUCCESSFUL  PRR. 

SLIDE  #9  ON 

•  PRODUCIBILITY 

•  WHAT  IS  THE  CONTRACTOR'S  MAKE  OR  BUY  STRUCTURE? 

•  WHAT  CONTROLS  ARE  MAINTAINED  ON  THE  SUBCONTRACTOR/VENDOR  NETWORK? 

I  DUAL  SOURCING  OF  CRITICAL  SUPPLIERS  AND  CONSEQUENT  DUAL  TOOLING 

COSTS  FROM  A  RISK  ASSESSMENT  VIEWPOINT. 
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•  MANAGEMENT/LABOR  EXPIRATION  DATES  FOR  KEY  SUPPLIERS. 

•  HARD-TO-OBTAIN  (LONG  LEAD)  RAW  MATERIALS. 

•  MANUFACTURING  CYCLE  TIMES  -  LEAD  TIMES  -  QUEUING! 

SLIDE  #9  OFF 

SLIDE  #10  ON 

•  CAPITAL  INVESTMENT 

•  ACQUISITION  OF  N/C,  EDM  EQUIPMENT. 

•  GOVERNMENT  ACQUISITION  OF  CAPITAL  EQUIPMENT  -  V* *IS  A  VIS  OUTSIDE 
MANUFACTURING  COST  TRADEOFF. ...  DOWNSIDE  RISK. ...  RETURN  ON  INVESTMENT.... 
DOD  POLICY  QUESTIONS. 

•  IN  THIS  REGARD  IT  IS  IMPERATIVE  THAT  WE  KNOW  THE  CONTRACTOR'S 
COMMERCIAL  AND  FMS  BUSINESS  PLANS  IN  ADDITION  TO  THE  MILITARY  REQUIRE¬ 
MENTS. 

•  FOR  IF  WE  ARE  PARTNERS  IN  A  JOINT  VENTURE,  LET'S  BE  FULL  PARTNERS 
AND  ASSESS  THE  ENTIRE  BUSINESS  PLAN  IN  ORDER  TO  MAKE  A  COMPLETE 
MANUFACTURING  EVALUATION. 

SLIDE  #10  OFF 
SLIDE  #11  ON 

•  CURRENT  AND  PAST  EXPERIENCE:  LESSONS  LEARNED  FROM  PREVIOUS  PROBLEMS.... 
WHAT  WERE  OR  ARE  THE  UNDERLYING  REASONS  FOR  THE  PROBLEMS? ...  HAVE  THEY  BEEN 
CORRECTED?. . .CAN  THEY  BE  AVOIDED  IN  THE  FUTURE? 

•  WHAT  SHIFT  LEVELS  ARE  PLANNED?. . .WHAT  OVERTIME  CONSIDERATIONS  WERE 
ASSESSED? 

•  WHAT  IS  THE  MANUFACTURING  CAPACITY?. . .MACHINE  LOADING  LEVELS?. . .EXPANSION 
CAPABILITIES? 

•  WHAT  IS  THE  CONTRACTOR'S  CAPITAL  OUTLAY  PLANS? .. .THEIR  FIVE  YEAR  AND 
10  YEAR  PLANS? 

I  WHERE  IS  THE  CONTRACTOR'S  LABOR  DRAWN  FROM?... IS  IT  AVAILABLE? 
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•  WHAT  ARE  THE  TRAINING  PROGRAMS? . . .ARE  THEY  ADEQUATE! 

SLIDE  jV\  OFF 

SLIDE  £12  ON 

•  WHAT  LOGISTICAL  CAPABILITIES  DOES  HE  HAVE? ...  FIELD  ORGANIZATION?... 

SPARES  SUPPORT?.. .ADP  CAPABILITIES? 

•  REVIEW  OF  THE  CONTRACTOR’S  KEY  MANAGERS  ASSIGNMENTS  TO  THE  PROGRAM.... 

WHAT  IS  THE  MANAGEMENT  ORGANIZATIONAL  NETWORK? ...  HOW  DOES  IT  WORK?... WHAT 
ASSURANCE  DO  WE  HAVE  OF  MAINTAINING  CONTINUITY  OF  ASSIGNMENT  WITH  KEY  MANAGERS? 

•  HOW  IS  LAYOUT,  WORKFLOW,  METHODS  AND  STANDARDS  INSTITUTED  AND  MAINTAINED? 

•  HOW  IS  THE  TRANSPORTATION  AND  PACKAGING  SYSTEMS  MANAGEMENT  PLANNED? 

•  HOW  HAS  THE  CONTRACTOR  PLANNED  THE  ASSEMBLY  AREA(S)? 

SLIDE  #12  OFF 

SLIDE  #13  ON 

•  HOW  ARE  PROPERTY,  PLANT  CLEARANCE  AND  INDUSTRIAL  RESOURCES  FUNCTIONS 
ACCOMMODATED?. . .SURGE  PLANNING  CAPABILITIES? 

«  ASPO 

•  GFP 

•  GFE 

I  TOOLING  -  NEED  CLASSIFICATION ....  PRIME  VENDOR. .. .RATE. .. .COMPONENT 

IMPROVEMENT. 

SLIDE  #13  OFF 

SLIDE  #14  ON 

I  ADEQUACY  AND  CAPACITY  OF  TEST  AREAS  AND  ASSOCIATED  EQUIPMENT. 

0  VENDORS  -  QPL,  ETC. 

•  HAS  ANY  PLANNING  BEEN  DONE  FOR  TRANSITION  AND  DOVE  TAILING  WITH  SHIFTS 


IN  THE  MAKE  OR  BUY  STRUCTURE. 


•  IS  c2  IS  A  CONSIDERATION. .. .IF  IT  IS  APPROPRIATE  -  REVIEWS. 

•  FLIGHT  SAFETY  PROVISIONS. 

SLIDE  #14  OFF 

SLIDE  #15  ON 

•  HOW  DOES  THE  CONTRACTOR  ACCOMPLISH  AND  MANAGE  HIS  VERTICAL  AND  HORIZONTAL 
PLANNING? 

•  HAS  PLANNING  INCLUDED  WORK  AROUNDS  FOR  BOTTLENECK  SCENARIOS? 

•  HOW  DOES  THE  CONTRACTOR  MANAGE  "SHOW  STOPPERS." 

DOES  THE  CONTRACTOR  HAVE  A  MASTER  PLAN  INTEGRATED  TO  LOWER  LEVELS?... 

WITH  A  DISCIPLINED  PROGRESS  TRACKING  SYSTEM? .. .WHAT  IS  THE  FREQUENCY  OF 
REVIEW  AND  UPDATE?. . .CORRECTIVE  ACTIONS? ...  DOES  IT  PINPOINT  RESPONSIBILITIES? 

•  ARE  PROCEDURES  DOCUMENTED? ...  HOW  ARE  THEY  CONTROLLED  AND  CHANGED?... 

WHAT  COMPLIANCE  TECHNIQUES  ARE  UTILIZED?. . .WATCH  OUT  FOR  THE  SHORT  CUTS, 
PARTICULARLY  THE  DIRE  EMERGENCY  ONES. 

SLIDE  #15  OFF 
SLIDE  #16  ON 

•  TOP-LEVEL  MANAGEMENT  INVOLVEMENT  AND  COMMITMENT. 

•  E.G.  OPEN  ACTION  ITEMS  AT  CONCLUSION  OF  THE  PRR. 

•  COMMUNICATIONS _ MATRIX  MANAGEMENT _ NAVY  AND  CONTRACTOR _ FEEDBACK 

LOOP  TO  PROGRAM  PEOPLE. 

•  ALL  THE  KNOWLEDGE  IN  THE  WORLD  WON'T  DO  ANYTHING  IF  IT'S  NOT 
COMMUNICATED  BACK  TO  PROGRAM  MANAGERS. 

•  HOW  WILL  THE  INTEGRATION  OF  THE  AIRCRAFT  ENGINE  MANUFACTURE  TAKE  PLACE 

WITH  THE  SYSTEMS'  CONTRACTOR  AND  AIRFRAMER? 

•  CONTRACTS'  PEOPLE  ARE  IMPORTANT! .. .THEY  KEEP  AS  ALL  HONEST! .. .STAY 

WITHIN  THE  FOUR  CORNERS  OF  THE  CONTRACT  OR  WORK  WITH  THE  PROGRAM  MANAGER  TO 

CHANGE  THE  CONTRACT. 
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•  BEWARE  OF  CONSTRUCTIVE  CHANGES ! 

•  BEWARE  OF  PRE-CONTRACT  COST! 

SLIDE  #16  OFF 

SLIDE  #17  ON 

•  ADVANCE  FUNDING.... LONG-LEAD  ITEMS. .. .QUEUING! 

•  KEEP  THE  COST  DOWN! 

•  KEEP  THE  QUALITY  UP! 

•  ENGINEERING  CHANGES  AND  CONFIGURATION  MANAGEMENT. 

•  COMPLIANCE  TO  DRAWINGS  AND  SPECIFICATIONS. 

SLIDE  #17  OFF 

SLIDE  #18  ON 

•  MRB,  REWORK,  SCRAP  -  ACTIVITIES  CAN  BE  DIAGNOSTIC  INDICATIONS  OF 
PROBLEM  AREAS.... OR  IF  THERE  IS  LITTLE  TO  NONE  HAS  IT  GONE  UNDERGROUND? 

SLIDE  m  OFF 

SLIDE  m  ON 

•  IN  CONCLUSION  THEN,  THE  PLANT  OFFICE  MUST  BE  AWARE  OF  THESE  SIGNIFICANT 
AREAS  OF  CONCERN.... MUST  ACCOMPLISH  THE  NECESSARY  INTERNAL  AND  EXTERNAL 
COORDINATION.... DEVELOP  AN  EXECUTIVE  BRIEF  FOR  THE  GOVERNMENT  MEETING 
SURFACING  ACTUAL  AND  POTENTIAL  PROBLEM  AREAS  AND  .PARTICIPATE  ACTIVELY  IN  A 
CRITICAL  REVIEW  OF  THE  PROGRAM. 

SLIDE  #19  OFF 

•  FINALLY,  LET'S  CLOSE  WITH  A  QUOTE  FROM  JOHN  LOCKE. 

"I  ATTRIBUTE  THE  LITTLE  I  KNOW  TO  MY  NOT  HAVING  BEEN  ASHAMED  TO 
ASK  FOR  INFORMATION,  AND  TO  MY  RULE  OF  CONVERSATION  WITH  ALL  DESCRIPTIONS 
OF  MAN  ON  THOSE  TOPICS  THAT  FORM  THEIR  OWN  PECULIAR  PROFESSIONS  AND  PURSUITS.* 
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NAVPRQ  LYNN 


•  WE  WHO  PARTICIPATE  IN  THE  PRODUCTION  READINESS  REVIEW  PERFORM 
IN  THE  ROLE  OF  A  CRITIC 


NAVPRQ  LYNN 

|  PURPOSE  OF  THE  REVIEW 

•  MINIMIZE  FUTURE  RISKS 

•  MBO 

•  ATTITUDE 
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NAVPRO  LYNN 


•  PLANNING 

•  SIGNIFICANT  REVISIONS  TO  DOD  DIRECTIVES  5000.1  AND  5000.2 

•  STREAMLINE  ACQUISITION  PROCESS 

•  PINPOINT  RESPONSIBILITIES 


NAVPRO  LYNN 

•  DOD  DIRECTIVE  5000.1 

•  AFFORDABILITY 

•  ACQUISITION  TIME 

•  STANDARDIZATION 

I  LOGISTIC  SUPPORTABI LITY 
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NAVPRO  LYNN 


•  INITIAL  PLANNING  STATEMENT  (IPS) 

•  COST  EFFECTIVENESS  ANALYSIS 

•  ORGANIZATIONAIVOPE  RATIONAL  CONCEPT 

•  CONFIGURATION  MANAGEMENT 
I  COST 

•  PRODUCTION  -  SECOND  SOURCE 

•  SOURCE  APPLICATION  OF  FUNDS 


NAVPRO  LYNN 

I  IPS  CHANGES  (CONTINUED) 

I  RELIABILITY  AND  MAINTAINABILITY 

•  QUALITY 

•  TRAINING 

•  FACILITIES 

•  COMPUTER  RESOURCES 

•  ENERGY 

•  HEALTH  AND  SAFETY 
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NAVPRO  LYNN 


I  COORDINATION 

•  INTERNAL 

•  EXTERNAL 


NAVPRO  LYNN 

•  INITIAL  GOVERNMENT  MEETING 
I  EXECUTIVE  SUMMARY 

•  SURFACE  PROBLEM  AREAS 

•  SINGLE  SPOKESPERSON 

•  NAVPRO 'S  CORPORATE  KNOWLEDGE 
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HAVPRO  LYNN 


•  PRODUCIBILITY 

•  MAKE  OR  BUY  STRUCTURE 
I  SUBCONTRACTOR/VENDOR 

•  DUAL  SOURCING 

•  MANAGEMENT/LABOR  EXPIRATION  DATES 

•  HARD-TO-OBTAIN  (LONG  LEAD)  ITEMS 

•  MANUFACTURING  CYCLES 


NAVPRO  LYNN 


I  CAPITAL  INVESTMENT 

•  NUMERICAL  CONTROL  AND  ELECTRONIC  DISCHARGE  EQUIPMENT 

•  GOVERNMENT  OWNERSHIP 

•  DOD  POLICY  QUESTIONS 

+  CONTRACTOR'S  TOTAL  BUSINESS  PLANS 
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NAVPRO  LYNN 


I  CURRENT  AND  PAST  EXPERIENCE 
I  SHIFT  LEVELS 
I  MANUFACTURING  CAPACITY 

•  CONTRACTOR'S  FIVE/TEN  YEAR  PLANS 
I  LABOR  DRAW 

•  TRAINING  PROGRAMS 


NAVPRO  LYNN 

•  LOGISTICAL  CAPABILITIES 

•  KEY  MANAGERS 

•  LAYOUT/WORKFLOW/METHODS  AND  STANDARDS 

•  TRAFFIC  AND  PACKING  MANAGEMENT 

•  ASSEMBLY  AREAS 
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NAVPRO  LYNN 


I  PROPERTY/PLANT  CLEARANCE/INDUSTRIAL  RESOURCES 

•  ASPO  -  SURGE  PLANNING 

•  GFP 

•  GFE 
•  TOOLING 

•  NEED  AND  CLASSIFICATION 

•  PRIME/VENDOR 

•  RATE 

I  COMPONENT  IMPROVEMENT 


NAVPRO  LYNN 

•  TEST  EQUIPMENT 

•  VENDOR  QPL 

•  PLANNING  ELASTICITY  FOR  CHANGES  IN  MAKE  OR  BUY  SHIFTS 

•  C2  -  REVIEWS 

•  FLIGHT  SAFETY  REQUIREMENTS 


167 


NAVPRO  LYNN 


I  VERTICAL  AND  HORIZONTAL  PLANNING 
I  WORKAROUNDS 
•'  SHOW  STOPPERS 
•  MASTER  PLAN 

I  DOCUMENTATION  OF  PROCEDURES 


NAVPRO  LYNN 

I  TOP-LEVEL  MANAGEMENT  INVOLVEMENT  AND  COMMITMENT 
I  COMMUNICATIONS 

I  INTEGRATION  WITH  SYSTEMS  AND  AIRFRAMERS 

I  CONTRACTING  OFFICER  KEPT  INFORMED 

I  BEWARE  OF  CONSTRUCTIVE  CHANGES 

•  BEWARE  OF  PRE-CONTRACT  COSTS 
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NAVPRO  LYNN 


•  ADVANCE  FUNDING/LONG-LEAD  ITEMS/QUEUING 

•  KEEP  IHE  COSI  DOM! 

•'  KEEP  IHE  QUALITY  UP! 

•  ENGINEERING  CHANGES/CONFIGURATION  MANAGEMENT 

•  COMPLIANCE  TO  DRAWINGS  AND  SPECIFICATIONS 


NAVPRO  LYNN 

•  MATERIAL  REVIEW  BOARD  (ACTIONS) 

•  REWORK 

•  SCRAP 
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NAVPRO  LYNN 

•  CONCLUSION 

•  SIGNIFICANT  AREAS  OF  CONCERN 
I  COORDINATION 

•  INTERNAL 
I  EXTERNAL 

•  EXECUTIVE  SUMMARY 

•  SURFACE  PROBLEM  AREAS 

I  PARTICIPATE  ACTIVELY  IN  THE  REVIEW 
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CRUISE  MISSILES  PRODUCTION  READINESS 


19-20  NOVEMBER  1980 

MAJOR  DON  ALDUCIN 
DIRECTOR,  PRODUCTION  DIVISION 
JOINT  CRUISE  MISSILES  PROJECT  OFFICE 


SLIDE  1 
TITLE 

GOOD  MORNING,  I'M  MAJOR  DON  ALDUCIN,  DIRECTOR  OF  PRODUCTION,  JOINT  CRUISE  MISSILES  PROJECT, 
(JCMP)  WASHINGTON,  D.C.  ON  BEHALF  OF  RADM  W.  LOCKE  AND  ALL  THE  MEMBERS  OF  THE  JCMP  I 
APPRECIATE  SHARING  WITH  YOU  THE  CRUISE  MISSILES  PRODUCTION  READINESS  APPROACH  AND  LESSONS 
LEARNED. 

I  REMEMBER  APPLYING  FOR  AIR  FORCE  OFFICER  TRAINING  SCHOOL  AND  THE  SERGENT  ASKING,  "DID  YOU 
COMPLETE  HIGH  SCHOOL?"  "YES,  SIR,  "  I  SAID,  "  I  GRADUATED  CUM  LAUDE  FROM  COLLEGE,  AND  HAVE 
COMPLETED  SEVERAL  HOURS  OF  GRADUATE  STUDIES."  THE  SERGEANT  NODDED,  REACHED  FOR  A  RUBBER 
STAMP,  AND  SLAPPED  IT  ON  THE  QUESTIONNAIRE.  IT  CONSISTED  OF  A  SINGLE  WORD:  "LITERATE." 

I  GUESS  BECAUSE  I  WAS  CLASSED  BY  THE  AIR  FORCE  AS  "LITERATE"  IT  ALSO  QUALIFIED  ME  TO  SERVE 
AS  A  PRR  TEAM  CHIEF. 

THIS  MORNING  I'M  GOING  TO  PROVIDE  A  CAPSULE  SUMMARY  OF  THE  JCMP  ORGANIZATION,  HOW  THE  JOINT 
OFFICE  WAS  ESTABLISHED,  ITS'  LINE  OF  AUTHORITY,  THE  EXCOM,  AN  OVERVIEW  OF  THE  FAMILY  OF 
CRUISE  MISSILES,  AND  THEIR  CHARACTERISTICS.  HOWEVER,  I  WILL  PRINCIPALLY  CONCENTRATE  ON: 
PRRs/PLANN ING  FOR  RATE  (PFRRs),  GUIDELINES,  TEAM  STRUCTURE,  AREAS  EVALUATED,  ASSESSMENT 
METHODS,  INCREMENTAL  SCHEDULE,  SUBCONTRACTOR  PRR  PROGRAM,  CONTRACTORS  GEOGRAPHIC  DISTRIBU¬ 
TION,  PRR/PFRR  FLOW  CYCLE,  AND  I'LL  CLOSE  MY  TALK  CONCENTRATING  ON  LESSONS  LEARNED  FROM  THE 
NUMEROUS  CONDUCTED  TO  DATE. 


JOINT  CRUISE  MISSILES  PROJECT  OFFICE 


SLIDE  2 

JCMPO  ORGANIZATION 


•  THE  JOINT  CRUISE  MISSILES  PROJECT  OFFICE  (JCMPO)  IS  A  DOD  CHARTER  JOINT  NAVY  AND  AIR  FORCE 
PROJECT,  THERE  ARE  A  TOTAL  OF  2Z£  EMPLOYEES.  AIR  FORCE  -  MILITARY  Z4  CIVILIANS  IS, 

NAVY  -  MILITARY  24  CIVILIANS  15Q.  RADM  WALTER  LOCKE  IS  THE  DIRECTOR  AND  COL,  ALLAN  CHASE 
IS  THE  DEPUTY  DIRECTOR.  THE  JCMPO  IS  LOCATED  IN  CRYSTAL  CITY  ON  THE  11TH/12TH  FLOORS  OF 
NATIONAL  CENTER  ONE. 
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PROGRAM  DIRECTION 


-  ESTABLISHED  JCMPO  -  NAVY  LEAD 

-COMMONALITY  IN  ENGINE,  NAVIGATION/ 
GUIDANCE,  WARHEAD 

•  30  SEPTEMBER  1977  DDR&E  MEMORANDUM 

-  RECONSTITUTED  JCMPO 

-  DIRECTED  ALCM  COMPETITION 

-  ESTABLISHED  EXECUTIVE  COMMITTEE 


SLICE  3 

PROGRAM  DIRECTION 

•  THE  DSARC  II  ESTABLISHED  THE  JCMPO,  DESIGNATING  THE  USN  AS  THE  LEAD  SERVICE,  THIS  DSAP.C  II 
DIRECTED  THAT  JCMPO  STRIVE  FOR  COMMONALITY  IN  THE  ENGINES,  NAVIGATION/GUIDANCE  AND  WARHEAD 
FOR  THE  AIR  FORCE  AND  NAVY  MISSILES  UNDERGOING  FULL  SCALE  ENGINEERING  DEVELOPMENT,  ALSO 
DIRECTED  THE  DEVELOPMENT  OF  GLCM  TO  MEET  AIR  FORCE  REQUIREMENT  USING  NAVY  TOMAHAWK  MISSILE, 

•  THE  DDR&E  MEMORANDUM  DATED  30  SEPTEMBER  1977  RECONSTITUTED  THE  JCMPO  AND  LOCATED  THE  ALCM/ 
GLCM/SLCM  PROGRAMS  WITHIN  THE  JCMPO  IN  WASHINGTON,  D.C.  THIS  MEMORANDUM  DIRECTED  A  COMPETI¬ 
TION  OF  THE  ALCM  AND  ESTABLISHED  THE  EXECUTIVE  COMMITTEE. 

•  THE  ALCM  DSARC  III  CONVENED  IN  MAY  1980  AND  DIRECTED  THAT  SYSTEM  INTO  FULL  RATE  PRODUCTION, 
AND  THE  ALCM  AIRFRAME  WAS  DIRECTED  BACK  TO  THE  AIR  FORCE'S  AERONAUTICAL  SYSTEMS  DIVISION 
FOR  PRODUCTION  MANAGEMENT;  HOWEVER,  THE  COMMON  ENGINE  AND  NAVIGATION/GUIDANCE  SYSTEMS  WERE 
DIRECTED  TO  BE  CONTINUED  TO  BE  MANAGED  EY  THE  JCMPO  TO  INSURE  THAT  MAXIMUM  COMMONALITY  BE 
MAINTAINED  IN  THE  SUBSYSTEMS  FOR  ALL  CRUISE  MISSILES. 
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ORGANIZATIONAL  LINES  OF 
AUTHORITY 


SLIDE  4 

ORGANIZATIONAL  LINES  OF  AUTHORITY 


•  THE  JCMPO  IS  A  DOD  CHARTERED  ORGANIZATION.  THE  NAVY  HAS  BEEN  DIRECTED  TO  PROVIDE  ADMINIS¬ 
TRATIVE  SUPPORT.  THE  ORGANIZATION  IS  ADMINISTRATIVELY  ASSIGNED  TO  NAVMAT,  DESIGNATED  JPM-3; 
HOWEVER,  RADM  LOCKE  RECEIVES  HIS  COMMAND  DIRECTION  FROM  ADM  WHITTLE,  CHIEF  OF  NAVMAT  FOR 
NAVY  SYSTEMS  AND  GENERAL  SLAY,  COMMANDER  AIR  FORCE  SYSTEMS  COMMAND  FOR  AIR  FORCE  SYSTEMS. 

ALL  FISCAL  AND  PROGRAMMATIC  DIRECTION  COMES  FROM  THE  EXECUTIVE  COMMITTEE  (EXCOM). 
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JOINT  CRUISE  MISSILES  PROJECT 
EXCOM 


MEMBERSHIP: 

OSD 

UNDER  SECDEF  (R&E)  CHAIRMAN 
AST  SECDEF  (PA&E) 

AST  SECDEF  (COMPT) 


NAVY  AIR  FORCE  ARMY 

VCNO  VCSAF  AST  SA  (RD&A) 

AST  SECNAV  (RE&S)  AST  SECAF  (RD&L) 

CHNAVMAT  CMDR,  AF  SYSCOM 


SLIDE  5 
EXCOM 


•  THE  EXCOM  MEETS  PERIODICALLY,  BUT  NOT  LESS  THAN  EVERY  TWO  MONTHS  TO  REVIEW  THE  PROGRAM 
STATUS  OF  THE  DEVELOPMENT  AND  PRODUCTION  PROGRESS  OF  ALL  THE  CRUISE  MISSILES  PROGRAMS. 
THE  EXCOM  IS  CHAIRED  BY  DR,  BILL  PERRY,  UNDER  SECRETARY  RESEARCH  AND  ENGINEERING  AND 
THOSE  REPRESENTED  ON  THE  CHART. 
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CRUISE  MISSILE  FAMILY 


r— —  SLCM - -  ■  —I 

p - GLCM - - 

TOMAHAWK 

r  -  _  _ 

1 

ANTISHIP  LAND  ATTACK 


- ALCM - - - 

(FLY-OFF  COMPETITION) 


SLCM:  ^ 

SEA-LAUNCHED  CRUISE  MISSILE 
(GENERAL  DYNAMICS) 

@  BGM-109B 
@  BGM-109C 
0  BGM-109A  (W-80) 

"TOMAHAWK":  NAME  FOR 


GLCM:  * 

GROUND  LAUNCHED 
CRUISE  MISSILE 
(GENERAL  DYNAMICS) 

0  BGM  109G  (W  84) 


©  AGM-109  (W-80): 

GENERAL  DYNAMICS 
®  AGM-86B  (W-80):  BOEING 


GENERAL  DYNAMICS  SLCM/GLCM  CRUISE  MISSILES 


AGM-109  AGM86B 


ALCM; 

AIR  LAUNCHED  CRUISE  MISSILE 


N:  NUCLEAR  ARMED  C:  CONVENTIONAL  ARMED 


SLIDE  6 

CRUISE  MISSILES  FAMILY 


•  THERE  ARE  PRESENTLY  12  DIFFERENT  VARIANTS  FOR  THE  CRUISE  MISSILES  EITHER  UNDER  DEVELOPMENT  OR 
IN  FULL  PRODUCTION,  THE  BOEING  AGM-86B  IS  ONLY  SYSTEM  PRESENTLY  IN  RATE  PRODUCTION.  EACH  OF 
THESE  SYSTEMS  ARE  BRICK-BAT  DX-A2  RATED.  (HIGHEST  NATIONAL  DEFENSE  PRIORITY  RATING) 


176 


CRUISE  MISSILE 


UNMANNED,  SELF-PROPELLED 
GUIDED  WEAPON-DELIVERY  VEHICLE 
WHICH  SUSTAINS  FLIGHT  THROUGH 
THE  USE  OF  AERODYNAMIC  LIFT 
OVER  MOST  OF  ITS  FLIGHT  PATH. 


SLIDE  7 

CRUISE  MISSILE 

•  UNMANNED  SELF  PROPELLED  GUIDED  WEAPON  WITH  A  SUBSONIC  F107  GAS  TURBINE  ENGINE  WITH  600 
LBS  OF  THRUST.  THE  RANGE  IS  IN  EXCESS  OF  1500  MILES  OR  2,500  KILOMETERS. 
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U.S.  CRUISE  MISSILE 
CHARACTERISTICS 


•  SPEED  AND  RANGE  -  SUBSONIC,  LONG  RANGE 


•  SURVIVABILITY  -DIVERSIFIED  SURVIVABLE 

LAUNCH  PLATFORMS 


•  PENETRATIVITY  -  LOW  ALTITUDE  CRUISE, 

SMALL  OBSERVABLES 


•  ACCURACY  -  NUCLEAR,  LOW  COLLATERAL 

DAMAGE;  CONVENTIONAL 
EFFECTIVE 


SLIDE  8 

CHARACTERISTICS 
AS  OUTLINED 
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CRUISE  MISSILE  REQUIREMENTS 


SLIDE  9 

PRODUCTION  QUANTITIES 

ALCK  -  3583 

SLCM  -  614 

GLCK  -  610 

MRASM  -  25Dtt 
TOTAL  -  8307 
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CRUISE  MISSILE  PRODUCTION  READINESS 
REVIEW  (PRR)  OVERVIEW 


•  PRR/PFRR  OBJECTIVES 

•  GUIDELINES 

•  TEAM  STRUCTURE 

•  AREAS  EVALUATED 

•  ASSESSMENT  METHOD/CODE 

•  PRR/PFRR  INCREMENTAL  SCHEDULE 

•  MAJOR  SUBCONTRACTOR  PRRs 

•  GEOGRAPHIC  DISTRIBUTION 

•  FLOW  CYCLE 

•  LESSONS  LEARNED 


SLIDE  10 
OVERVIEW 

AS  OUTLINED 
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CRUISE  MISSILE  (CM)  PRODUCTION 
READINESS  REVIEW  OBJECTIVES 


•  ENSURE  THAT  THE  CM  CONTRACTORS,  EACH  ASSOCIATE 
CONTRACTOR,  AND  KEY  SUBCONTRACTORS  ARE 
READY  FOR  PRODUCTION  THAT  SATISFIES  IDENTIFIED 
PERFORMANCE  AND  SCHEDULE  REQUIREMENTS 


•  ENSURE  ALL  SIGNIFICANT  PRODUCTION  ENGINEERING 
PROBLEMS  ENCOUNTERED  DURING  DEVELOPMENT 
HAVE  BEEN  RESOLVED 


•  ENSURE  THE  ACCOMPLISHMENT  OF  ADEQUATE 

PLANNING  AND  PREPARATION  FOR  THE  TRANSITION 
FROM  DEVELOPMENT  TO  PRODUCTION 


SLIDE  11 
OBJECTIVES 


o  THESE  ARE  THE  THREE  MAJOR  OBJECTIVES  OF  THE  CRUISE  MISSILES  PRRs. 
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GUIDELINES  FOR  CONDUCTING 
REVIEWS 


•  JCMP  PRR/PFRR  PLAN 

•  DoDI  5000.38 

•  COORDINATED  AGENDAS 

•  JOINT  SERVICE  TEAMS 


SLIDE  12 
GUIDELINES 


•  THE  REASON  THE  JCMP  HAS  BEEN  AS  SUCCESSFUL  IN  ACHIEVING  MAXIMUM  PRODUCTION  READINESS  IS  BECAUSE 
OF  THE  DETAILED  PLANNING  THAT  HAS  BEEN  ACCOMPLISHED.  NUMEROUS  UPDATES  HAVE  BEEN  PREPARED  TO 
THE  JCMP  PRR/PFRR  PLAN.  THE  JCMP  FOLLOWS  THE  DODI  5000.38  ALMOST  TO  THE  LETTER.  THE  KEY 
FEATURE  OF  THE  CRUISE  MISSILE  PROCESS  IS  THAT  THERE  IS  A  DETAILED  COORDINATED  AGENDAS  WITHIN 
THE  PROJECT  AND  THEN  WITH  THE  CONTRACTOR  TO  INSURE  THERE  IS  MAXIMUM  UNDERSTANDING  OF  WHAT  IS 
TO  BE  REVIEWED.  THE  JCMP  HAS  BEEN  ABLE  TO  OBTAIN  SOME  OF  THE  BEST  INDUSTRIAL  CONSULTING 
EXPERTS  FROM  INDUSTRY  AND  THE  GOVERNMENT  TO  CONDUCT  THE  REVIEWS, 
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CRUISE  MISSILES  PRODUCTION 
READINESS  REVIEW  TEAM  STRUCTURE 


PRODUCTION 

ENGINEERING 


DIRECTOR  JCMPO 


DIRECTOR  PROJECT  X 

I 


NOTE:  DoD  PESO  OBSERVED 
ALL  KEY  PRRs 


TEAM  CHIEF  JCMPO 


I 


ASSISTANT  TEAM  CHIEF 
(CONTRACT  ADMINISTRATION 
ORGANIZATION) 


MANUFACTURING 

MANAGEMENT 


QUALITY 

ASSURANCE 


SUBCONTRACT/ 

MATERIAL 


-  JCMPO 

-  AIR  FORCE  MATERIAL  LAB 

-  NAVAL  WEAPONS  CENTER 

-  AIR  FORCE  SYSTEMS 
COMMANO 

-  AFSC  AERONAUTICAL 
SYSTEMS  0IVISI0N 

-  USN  PROOUCT  ENGINEERING 
SERVICES  OFFICE 

-  AFPRO 

-  OCAS 

-  SELECTEO  CONSULTANTS 


-  JCMPO 

-  AIR  FORCE  SYSTEMS 
COMMANO 

-  NAVAL  WEAPONS  CENTER 

-  AIR  FORCE  MATERIAL  LAB 

-  AFPRO 

-  OCAS 

-  ASO 

-  SELECTEO  CONSULTANTS 


-  JCMPO 

-  NAVAL  WEAPONS  CENTER 

-  SELECTEO  CONSULTANTS 

-  AIR  FORCE  SYSTEMS 
COMMANO 

-  AFPRO 

-  OCAS 


-  JCMPO 

-  SELECTED  CONSULTANTS 

-  AFPRO 

-  DCAS 


SOFTWARE 


JCMPO 

AFALO 

ALC 

AFPRO 

OCAS 


-  JCMPO 

-  ASO 

-  AFPRO 

-  DCAS 

-  NAVAL  WEAPONS 
CENTER 


* 


SLIDE  13 
TEAM  STRUCTURE 


•  THE  CHARTS  REFLECT  WHAT  A  TYPICAL  TEAM  LOOKED  LIKE  DURING  THE  ALCM  PRR  AT  THE  AIR  VEHICLE 
CONTRACTORS  REVIEW, 


•  THE  TEAMS  WERE  ANYWHERE  FROM  35  INDIVIDUALS  TO  A  MAXIMUM  OF  55  PEOPLE,  DOD  PESO  WAS  ALWAYS 
INVITED  TO  PARTICIPATE  AS  AN  OBSERVER  WHERE  DESIRED. 
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PRODUCTION  READINESS  REVIEWS 
KEY  AREAS  EVALUATED 


•  PRODUCTION  ENGINEERING 

-  FACILITIES  PLAN 

-  TOOLING  PROGRAM 

-  PRODUCT  FLOW 

-  PRODUCIBILITY  PROGRAM 

-  MANPOWER  PLANNING 

-  LOAD  CAPACITY  PLANNING 

•  MANUFACTURING  MANAGEMENT  SYSTEMS 

-  PRODUCTION  MANAGEMENT 
-WORK  MEASUREMENT 
-CONFIGURATION  MANAGEMENT 

-  SCHEDULING/PRODUCTION  CONTROL 


PRODUCTION  READINESS  REVIEWS 
KEY  AREAS  EVALUATED  (Cont’d) 


•  QUALITY  ASSURANCE 

-  Q.A.  SYSTEM 

-  Q.A.  PROGRAM  PLAN 

-  Q.A.  ORGANIZATION 

-  CORRECTIVE  ACTION  SYSTEM 

•  SUBCONTRACTS 

-  MAKE  OR  BUY  STRUCTURE 

-  PROCUREMENT  SYSTEMS 

-  SUBCONTRACTOR  PRODUCTION 
READINESS  REVIEWS 

-  SUBCONTRACT  MANAGEMENT 
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PRODUCTION  READINESS  REVIEWS 
KEY  AREAS  EVALUATED  (Cont’d) 


•  LOGISTICS 

-  TECH  ORDERS 

-  OPERATIONAL  SUPPORT 

-  LOGISTICS  SUPPORT  ANALYSIS 

-  PROVISIONING/SPARES 

•  SOFTWARE 

-OPERATIONAL/MISSION  PLANNING 

-  TEST  EQUIPMENT 
-SOFTWARE  INTEGRATION 

-  FLIGHT  TEST 


SLIDE  14/15/16 
AREAS  EVALUATED 

•  THE  NEXT  THREE  CHARTS  REFLECT  PRINCIPAL  AREAS  COVERED  BY  THE  PANEL  DURING  THE  PRR'S. 

•  THE  TYPICAL  CRUISE  MISSILE  PRR  WOULD  COMMENCE  ON  MONDAY  MORNING  AT  0800  HOURS  AT  THE 
CONTRACTORS'  PLANT.  A  GOVERNMENT  TEAM  WOULD  CONVENE  FOR  APPROXIMATELY  30  MINUTES  TO  DIS¬ 
CUSS  ADMINISTRATIVE  PROCEDURES.  THEN  AT  0830  HRS.  A  MEETING  WOULD  CONSIST  OF  ALL  PANELS 
AND  THE  CONTRACTOR  TO  GO  OVER  THE  LATEST  PROGRAM  STATUS  PRINCIPALLY  CONCENTRATED  ON  THE 
DEVELOPMENT  EFFORT  PRODUCTION  PLAN  IMPLEMENTATION.  THIS  FIRST  SESSION  WOULD  GENERALLY 
COVER  THREE  HOURS.  AFTER  LUNCH  THE  PANELS  WOULD  COVER  THEIR  ASSIGNED  AREAS.  MANY  PANELS 
WOULD  BE  JOINTLY  SCHEDULED  THROUGHOUT  THE  WEEK.  THE  DETAILED  REVIEWS  WOULD  BE  SCHEDULED 
TO  BE  COMPLETE  BY  1200  HOURS  ON  THURSDAY.  DURING  THAT  AFTERNOON  EACH  PANEL  WOULD  PRESENT 
THEIR  FINDINGS  TO  THE  TEAM  CHIEF.  THIS  WOULD  BE  IN  THE  FORM  OF  A  BRIEFING  COVERING  THE 
AREAS  EVALUATED.  IT  WOULD1  IDENTIFY  THE  CONTRACTORS  MAJOP  STRENGTHS  AND  WEAKNESSES.  EACH 
AREA  WOULD  BE  EVALUATED  AS  TO  THE  AMOUNT  OF  PROGRESS  MADE  FOR  THAT  REIVEW  WITH  AN  ASSESS¬ 
MENT  COLOR  CODE.  LATER  IN  THE  AFTERNOON  THE  BRIEF  WOULD  BE  REVIEWED  BY  THE  PROGRAM  MANAGER 
FOR  HIS  CONCURRENCE.  THE  CONTRACTORS  GRAPHICS  PERSONNEL  WOULD  BE  REQUESTED  TO  PREPARE  THE 
VUEGRAPHS  AMD  HARD  COPIES  OF  THE  PRESENTATION  THAT  EVENING.  FRIDAY  MORNING  THE  CHARTS  WOULD 
BE  REVIEWED  AT  ABOUT  0730  BY  EACH  PANEL  CHAIRMAN.  AT  0900  AN  EXECUTIVE  DE-BRIEFING  OF  THE 
RESULTS  OF  THE  PRR  WOULD  BE  GIVEN  TO  THE  TOP  EXECUTIVE  MANAGEMENT  -  PRESIDENT  AND  VICE 
PRESIDENT  LEVEL.  THIS  BRIEFING  WOULD  NORMALLY  BE  ATTENDED  BY  THE  DIRECTOR  OR  DEPUTY  DIRECTOR 
OF  THE  JCMP.  THIS  WOULD  BE  A  FORMAL  REVIEW.  QUESTIONS  WOULD  NORMALLY  NOT  ALLOWED  EXCEPT 
FOR  CLARIFICATION.  ALL  THE  DEFICIENCIES  WOULD  BE  WRITTEN  UP  ON  A  "REQUEST  FOR  ACTION"  FORM 
AND  CONEYED  TO  THE  CONTRACTOR  THRU  A  CONTRACTING  OFFICER  LETTER.  PRIOR  TO  DEPARTURE  FROM  THE 
CONTRACTORS  PLANT  ALL  ADMINISTRATIVE  ACTIONS  WOULD  HAVE  BEEN  COMPLETED  INCLUDING  A  WRITTEN 
NARRATIVE  EXECUTIVE  SUMMARY.  THIS  WOULD  BE  MAINTAINED  IN  A  PRR  CASE  FILE  WITH  COPIES  OF  ALL 
PRESENTATIONS,  REQUEST  FOR  ACTIONS,  AGENDAS,  SCHEDULES  EXECUTIVE  PANEL  SUMMARIES  AND  ANY  OTHER 
PERTINENT  DATA.  THIS  FILE  WOULD  BE  MAINTAINED  BY  THE  EXECUTIVE  OFFICER  -  NORMALLY  MY  PROGRAM 
COORDINATOR  OR  A  MILITARY  ADMINISTRATIVE  OFFICER. 
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ASSESSMENT  METHOD 


AREA 

INVESTIGATED 


SUBJECT 


PERCENT  COMPLETE 


20 


40 


60 


100 


ASSESSMENT 

CODE 
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ASSESSMENT  CODE 


-CONTINGENCY  PLANNING  AND  IMPLEMENTATION  HAVE  ADDRESSED  ALL  IDENTIFIED 
RISKS 


-  NOTHING  MAJOR  HAS  BEEN  IDENTIFIED  THAT  WILL  PREVENT  THE  ACHIEVEMENT  OF 
DESIRED  RESULTS 


-  ADVERSE  IMPACTS  HAVE  BEEN  IDENTIFIED  THAT  WOULD  POSSIBLY  PREVENT  THE 
ACHIEVEMENT  OF  DESIRED  RESULTS  WITHOUT  CORRECTIVE  ACTION  . 


-SIGNIFICANT  ADVERSE  IMPACTS  HAVE  BEEN  IDENTIFIED  THAT  WILL  IMPACT  THE 
ACHIEVEMENT  OF  DESIRED  RESULTS  WITHOUT  IMMEDIATE  MANAGEMENT  ATTENTION 


4 

O 

<> 


PROGRESS  SINCE  THE  LAST  PRR  HAS  BEEN  BETTER  THAN  EXPECTED 
PROGRESS  SINCE  THE  LAST  PRR  HAS  BEEN  SATISFACTORY 
PROGRESS  SINCE  THE  LAST  PRR  HAS  BEEN  LESS  THAN  SATISFACTORY 


SLIDE  17/18 

ASSESSMENT  METHOD/CODES 

•  THIS  IS  HOW  WE  USE  THE  BAROMETER  SCALE  AND  COLOR  CODE  ASSESSMENTS. 
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CRUISE  MISSILES 
PRR/PFRR  SCHEDULE 
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SLIDE  19 

1978/79/80  PRR  SCHEDULE 


•  IT  SEEKS  LIKE  I'VE  SPENT  KORE  TIME  IN  AIRPORTS  AND  MOTELS  THAN  a  HOME.  WE  ACCOMPLISHED 
26  INCREMENTAL  PRRs  PRIOR  TO  THE  ALCM  DSARC  III 
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JCM-25  PRR/PFRR  SCHEDULE 
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NOTE  1.  CWCS  MINI  REVIEW 


LEGEND 

OPFRR 

OPRR 

□  MM/PCR 

SLIDE  20 

1980/81  SCHEDULE 

•  LOOKS  LIKE  IT  IS  GOING  TO  BE  A  BUSY  WINTER 
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MAJOR  SUBCONTRACTOR  PRRs 


•  ASSOCIATE  CONTRACTORS  RESPONSIBLE 
FOR  CONDUCTING  PRRs  AT  SELECTED 
SUBCONTRACTORS 


•  CRITERIA  AND  METHODOLOGY  REVIEWED 
BY  GOVERNMENT  PRR  TEAM 


•  SELECTED  GOVERNMENT  PRR  TEAM 
PERSONNEL  ATTENDED  SUBCONTRACTOR 
PRRs  AS  OBSERVERS/PARTICIPANTS 


SLIDE  21 

MAJOR  SUBCONTRACTOR  PRRs 


•  EACH  OF  THE  ASSOCIATE  CONTRACTORS  WERE  RESPONSIBLE  FOR  CONDUCTING  PRRs  AT  THE  CRITICAL 
AND  MAJOR  SUBCONTRACTORS  TO  ASSURE  THE  READINESS  OF  ALL  SUPPLIERS.  THE  PROGRAM  OFFICE 
RECEIVED  THE  AGENDAS  AND  PARTICIPATED  DURING  MANY  REVIEWS.  MOST  OF  THE  REVIEWS  WERE 
DELEGATED  TO  THE  CONTRACT  ADMINISTRATION  OFFICE  TO  HANDLE  THIS  PORTION  OF  THE  PRR  PRO¬ 
GRAM. 
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GEOGRAPHIC  DISTRIBUTION 
CM  ASSOCIATE/MAJOR  SUBS 
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□  JCMPO 


SLIDE  22 

GEOGRAPHIC  DISTRIBUTION 

I  THESE  ARE  THE  MAJOR  ASSOCIATE  AND  CRITICAL  MAJOR  SUBCONTRACTORS. 
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PRODUCTION  READINESS 
FLOW  CYCLE 


STATEMENT  DF  WORK  PRR  REVIEW  DSARC  PRR 
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SLIDE  23 

PRODUCTION  READINESS  FLOW  CYCLE 

•  THIS  SLIDE  ATTEMPTS  TO  SHOW  HOW  THE  JCMP  STRUCTURED  THE  PROCESS.  FIRST,  EACH  ASSOCIATE 
CONTRACTORS  CONTRACT  HAD  REQUIREMENTS  IN  THE  STATEMENT  OF  WORK  COVERED.  MOST  OF  THE 
AREAS  WERE  REQUIRED  TO  BE  STATED  AS  TO  THE  CONTRACTORS  PLAN  FOR  ACTION  IN  THE  CONTRACT 
DATA  REQUIREMENTS  LIST  (CDRL)  PRODUCTION  PLAN.  THE  NEXT  THING  WAS  TO  PUT  TOGETHER  A  PRR 
TEAM  AND  THEN  INSURE  THAT  THE  AREAS  THE  DSARC  III  WAS  INTERESTED  IN  WERE  COVERED.  THE 
LAST  AREA  IS  THE  ONE  THAT  WE  ACCOMPLISHED  EVEN  AFTER  THE  DSARC  III.  THAT  IS  THE  PLANNING 
FOR  RATE  REVIEWS  (PRFFs) .  THIS  REVIEW  IS  TO  INSURE  THAT  RATE  IS  ATTAINED.  IN  FACT,  I 
LEAVE  THIS  EVENING  TO  ATTEND  THE  EXECUTIVE  DE-BRIEFING  AT  THE  BOEING  COMPANY  TOMORROW  MORN¬ 
ING  ON  THE  AGM-86B.  PFRR  WHICH  IS  BEING  CONDUCTED  BY  ASD.  THIS  WILL  BE  A  FIRST  FOR  ME. 

I  GET  TO  JUST  LISTEN  AS  AN  OBSERVER  FROM  THE  JCMPO. 
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LESSONS  LEARNED  FROM  PRRs 


•  SIZE  OF  TEAMS  MUST  BE  LIMITED 

•  DE-EMPHASIZE  IMPORTANCE  OF  THE  NUMBER  COUNT  OF  RFRs 

•  PRRs  IMPROVED  COMMUNICATIONS  WITHIN  THE  CONTRACTOR’S 
ORGANIZATION 

•  INDUSTRIAL  CAPACITY  TO  SUPPORT  DEPLOYED  SYSTEM 

•  CONTRACTOR  MANPOWER/EQUIPMENT  PREDICTION  METHODS  AND 
DATA  BASE  SHOULD  BE  ASSESSED 

•  USEFULNESS  OF  SELECTED  INDUSTRIAL  EXPERTS  WITH  PREVIOUS 
EXPERIENCE  IN  PRIVATE  INDUSTRY 

•  JCMP  ABILITY  TO  USE  BOTH  NAVY  AND  AIR  FORCE  RESOURCES 

•  TOP  MANAGEMENT  INVOLVEMENT  AT  BOTH  PROGRAM  OFFICE 
AND  CONTRACTOR 

•  PRODUCTION  PLANS  SHOULD  PROVIDE  BASIS  FOR  THE  PRR 
ASSESSMENT 

•  CONTRACTOR  COOPERATION  MAY  LEAD  TO  EXPOSING  RISK 


LESSONS  LEARNED  FROM  PRRs  (CONT’D) 


•  ACCOMPLISHMENT  OF  OBJECTIVES 


•  CONTRACTOR  PROGRESS  TOWARDS  PRODUCTION  READINESS 

•  CONTACT  WITH  THE  CONTRACTORS  MUST  BE  MAINTAINED 
BETWEEN  PRRs 


•  A  DETAILED  AGENDA  MUST  BE  PUBLISHED  IN  ADVANCE  OF 
EACH  PRR 


•  LAST  INCREMENTAL  PRR  MUST  BE  HELD  IMMEDIATELY  PRIOR 
TO  DSARC  III 


•  THERE  SHOULD  BE  CONTINUITY  OF  PANEL  CHAIRMEN 

•  MOST  PANEL  MEMBERS  MUST  BE  CAREFULLY  SELECTED  AS 
TO  QUALIFICATIONS 

•  EXPECTED  PERFORMANCE  STANDARDS  SHOULD  BE  DEFINED 
PRIOR  TO  PRRs 
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LESSONS  LEARNED  FROM  PRRs  (CONT’D) 


•  CONTRACTOR’S  DECISION  -  PROCEDURES/PROCESSES  DO  NOT 
NORMALLY  CONSIDER  PRODUCTION  COSTS 

•  PRR  TEAM  MUST  VALIDATE  THE  VU-GRAPH  PRESENTATIONS  OF 
DATA  THROUGH  HARDWARE  FLOOR  AUDITS 

•  MIL-STD-1528  "PRODUCTION  MANAGEMENT"  INTERNAL  AUDIT 
REQUIREMENT  MUST  BE  ACCOMPLISHED  BY  THE  CONTRACTORS 

•  DURING  PRRs,  MUST  HAVE  DAILY  CROSS-PANEL  COORDINATION 


SLIDE  24/25/26 
LESSONS  LEARNED 


•  ACCOMPLISHMENT  OF  OBJECTIVES. 

THE  INCREMENTAL  PRRs  DID  ENSURE  CRUISE  MISSILES  CONTRACTORS  WERE  READY  FOR  PRODUCTION  AND 
PERMITTED  THOROUGH  ASSESSMENT  OF  REMAINING  PROBLEMS. 

•  CONTRACTOR  PROGRESS  TOWARDS  PRODUCTION  READINESS. 

THE  INCREMENTAL  PRRs  ACCELERATED  OVERALL  CONTRACTOR  PROGRESS  TOWARDS  PRODUCTION  READINESS  BY 
EXPOSING  PROBLEM  AREAS  AND  THE  PROGRESS  MADE. 

•  CONTACT  WITH  THE  CONTRACTORS  MUST  BE  MAINTAINED  BETWEEN  PRRs. 

THE  BEST  PROGRESS  WAS  MADE  BY  THE  CONTRACTORS  WHEN  THERE  WERE  FOLLOW-UP  CONTACTS  BY  JCMP.  THIS 
ENSURED  THAT  CONTRACTOR  EFFORTS  WERE  APPLIED  ALONG  THE  PROPER  LINES  OF  DIRECTION.  CONVERSELY 
THERE  WAS  ONE  INSTANCE  WHEN  CONTACT  WAS  LOST  WHILE  THE  CONTRACTOR  DRASTICALLY  CHANGED  PRODUCTION 
FACILITY  PLANNING. 

•  A  DETAILED  AGENDA  MUST  BE  PUBLISHED  IN  ADVANCE  OF  EACH  PRR. 

WHEN  A  DETAILED  AGENDA  WITH  SPECIFIC  DIRECTION  WAS  PROVIDED  TO  THE  CONTRACTOR,  ALLOWING  PREPAR¬ 
ATION  TIME  FOR  THE  TOPICS  OF  INTEREST,  THE  PRR  PROVED  MORE  PRODUCTIVE  THAN  WITH  A  GENERALIZED 
AGENDA.  BOTH  GOVT  8  CONTRACTOR  MUST  UNDERSTAND  THE  AGENDA  ITEM  INTENT  OR  OBJECTIVE.  BROAD 
INITIAL  AGENDA  VS  CONCENTRATED  ON  WEAK  AREAS  I.E.  AFSCR  84-2. 

•  LAST  INCREMENTAL  PRR  MUST  BE  HELD  IMMEDIATELY  PRIOR  TO  DSARC  III. 

IF  THERE  IS  A  LONG  PERIOD  BETWEEN  THE  LA, ST  INCREMENTAL  PRR  AND  DSARC  III  DATA  PROVIDED  TO  DOD 
WILL  BE  INACCURATE. 

•  THERE  SHOULD  BE  CONTINUITY  OF  PANEL  CHAIRMEN 

IT  IS  NECESSARY  TO  HAVE  CONTINUITY  OF  PANEL  MEMBERS  TO  PROVIDE  A  CONSISTENT  EVALUATION.  THE 
PANEL  CHAIRMEN  SHOULD  BE  SELECTED  FROM  WITHIN  THE  PROGRAM  MANAGEMENT  ORGANIZATION  TO  PROVIDE 
DAY-TO-DAY  CONTACT  WITH  WORKING  LEVEL  PEOPLE  WITHIN  THE  PM, 
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SLIDE  24/25/26 
LESSONS  LEARNED  (CONT) 

•  MOST  PANEL  MEMBERS  MUST  BE  CAREFULLY  SELECTED  AS  TO  QUALIFICATION. 

•  EXPECTED  PERFORMANCE  STANDARDS  SHOULD  BE  DEFINED  PRIOR  TO  PRR. 

THE  ANTICIPATED  CONTRACTOR  PROGRESS  AND  STATUS  USED  FOR  THE  STANDARDS  OF  THE  ASSESSMENT 
SHOULD  BE  DEFINED  PRIOR  TO  EACH  PRR 

•  SIZE  OF  TEAM,  MEMBERS  MUST  BE  LIMITED  TO  WORKABLE  NUMBER  (5  NOT  50) 

•  DE-EMPHASIZE  IMPORTANCE  OF  THE  NUMBER  COUNT  OF  RFRs. 

IT  SHOULD  BE  IMPRESSED  ON  THE  CONTRACTORS  THAT  THE  NUMBER  OF  REQUESTS  FOR  RESPONSE  (RFRs) 
ISSUED  AS  A  RESULT  OF  THE  PRR  IS  NOT  AS  IMPORTANT  AS  THE  CONTENT  OF  THE  RFRs.  CONSOLIDATE 
WHENEVER  POSSIBLE  &  CONFINE  TO  MAJOR  ISSUES. 

•  PRRs  IMPROVED  COMMUNICATIONS  WITHIN  THE  CONTRACTORS  ORGANIZATION. 

BECAUSE  OF  THE  COMPREHENSIVE  AND  CENTRALIZED  NATURE  OF  THE  REVIEWS,  COMMUNICATIONS  BETWEEN 
THE  INTERNAL  FUNCTIONS  WERE  IMPROVED.  DUPLICATION  OF  WORK  AND  CROSS-PURPOSE  EFFORTS  WERE 
REDUCED, 

»  INDUSTRIAL  CAPACITY  TO  SUPPORT  DEPLOYED  SYSTEM  (REFURB/RECERT  DEPOT) 

THE  INCREASED  FABRICATION  AND  ASSEMBLY/TEST  LOAD  IMPOSED  BY  RECERTIFICATIONS  SHOULD  EE 
EVALUATED  BEFORE  DSARC  III. 

•  CONTRACTOR  MANPOWER/QU IPMENT  PREDICTION  METHODS  &  DATA  BASE  SHOULD  BE  ASSESSED. 

THE  METHODS  AND  DATA  BASE  USED  FOR  MANPOWER/ECU IPMENT  ACQUISITION  PLANNING  MUST  BE  EVALUATED 
AS  THE  BASIS  FOR  THE  PRODUCTION  PLAN. 

•  USEFULNESS  OF  SELECTED  INDUSTRIAL  EXPERTS,  PREVIOUSLY  IN  PRIVATE  INDUSTRY. 

THE  WEALTH  OF  PRODUCTION  EXPERIENCE  POSSESSED  BY  RETIRED  MANUFACTURING  EXECUTIVES  WITH  INDUS¬ 
TRIAL  BACKGROUND  WAS  VALUABLE  IN  EVALUATING  THE  CONTRACTORS  PRODUCTION  PLANS,  SYSTEMS, 
PROCEDURES 


SLIDE  24/25/26 
LESSONS  LEARNED  (CONT) 

•  JCMP  ABILITY  TO  UTILIZE  BOTH  NAVY  AND  AIR  FORCE  RESOURCES. 

CHINA  LAKE  PROVIDED  EXPERTISE  IN  SOLDERING  TECHNOLOGY,  ALSO  PROVIDED  THE  CHAIRMAN  FOR  THE 
PRODUCTION  ENGINEERING  PANEL.  AF  MATERIALS  LAB  ADVANCED  COMPUTER  AIDED  MANUFACTURING 
PRODUC IB  I LITY  PROGRESS  WITH  MANUFACTURING  TECHNOLOGY  PROJECTS. 

<>  TOP  MANAGEMENT  INVOLVEMENT  AT  BOTH  PROGRAM  OFFICE  AND  CONTRACTOR. 

•  PRODUCTION  &  PLANS  SHOULD  PROVIDE  BASIS  FOR  THE  PRR  ASSESSMENT. 

THEY  SHOULD  BE  CUSTOMIZED  TO  CDRL  DID  THAT  IS  FORMATED  TO  THE  PRR  AREAS  TO  BE  EVALUATED. 
THIS  DOCUMENT  MUST  BE  A  WORKING  PLAN  FOR  BOTH  THE  CONTRACTOR  AND  PROGRAM  OFFICE. 

•  CONTRACTOR  COOPERATION  MAY  LEAD  TO  EXPOSING  RISK. 

CERTAIN  DEPARTMENTS/FUNCTIONS  THAT  TEND  TO  BE  COOPERATIVE  SHOULD  NOT  BE  PENALIZED  WHEN 
RISK  IS  EXPOSED. 

•  CONTRACTORS  DECISION-PROCEDURES/PROCESSES  DO  NOT  NORMALLY  CONSIDER  PRODUCTION  COST  IN 
DESIGN-TO-COST  PROGRAMS,  PRODUCTION  COST  SHOULD  BE  A  KEY  FACTOR  IN  DECISIONS. 

•  PRR  TEAM  MUST  VALIDATE  THE  VUEGRAPH  PRESENTATIONS  OF  DATA  THRU  HARDWARE  FLOOR  AUDITS. 

•  MIL-STD-1528  "PRODUCTION  MANAGEMENT"  INTERNAL  AUDITS  REQUIREMENTS  MUST  BE  ACCOMPLISHED  BY 
THE  CONTRACTORS  IN  LIEU  OF  GOVT  PRR  TEAM  ACCOMPLISHING  THIS  FUNCTION. 

•  MUST  HAVE  DAILY  CROSS-PANEL  COORDINATION  (IN  MORNING  ONLY)  MEETING  TO  REVIEW  MAJOR  FIND¬ 
INGS  TO  AVOID  DUPLICATION  AND  ENHANCE  COHERENCY. 
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AIR  FORCE  OVERVIEW 
FOR 

DOD  PRR  CONFERENCE 
WPAFB  OHIO 
20  NOV  80 


Good  Morning : 

This  may  come  as  a  shock  to  you,  Mr.  Baldwin,  but  under  the  ground  rules  that 
we  should  delete  items  that  have  already  been  covered  -■  -  -  the  Air  Force  has 
nothing  to  present  this  morning.  So  as  soon  as  you  wrap  up  the  conclusions  we 
can  be  on  our  way  home.  ITve  always  known  the  tradition  or  habit  of  always 
putting  the  Air  Force  last  by  saying  Army,  Navy  and  Air  Force  would  be  a  break 
someday.  Glad  you  didn’t  break  tradition  and  go  in  alphabetical  order. 

Seriously  though,  the  excellent  presentations  we’ve  heard  from  the  Army,  Navy 
and  DCAS  make  it  most  difficult  to  see  "having  the  last  word"  as  being  an 
advantage  in  this  case  —  ~  as  our  wives  sometimes  think  and  certainly  saving 
the  best  to  last"  does  not  logically  apply  to  this  situation. 

To  get  down  to  business,  I  believe  it  would  be  most  helpful  to  your  understanding 
and  conserve  time  if  I  focus  on  the  unique  and  key  things  about  the  Air  Force 
in  regard  to  PRRs. 

1st.  The  Department  of  the  Air  Force  delegated  all  responsibility  and  authority 
to  HQ  AFSC  for  all  manufacturing  activities  related  to  weapon  systems  acquisition. 
This  was  accomplished  in  AFR  800-9  dated  1  Oct  79. 

2nd.  HQ  AFSC  spends  about  $14  billion  per  year  through  5  Buying  Divisions  and  one 
Administration  Division,  who  have  approximately  680  authorized  spaces  devoted 
to  looking  after  manufacturing  interests.  This  includes  approximately  400  in  HQ 
AF  Contract  Management  Division  at  Albuqurque,  New  Mexico  and  their  AFPROs  and 
about  280  in  the  buying  divisions.  Aeronautical  Systems  Division,  here  at  WPAFB 
has  the  most  with  165.  Electronic  Systems  Division  at  Kanscom  AFB,  MASS  has 
52,  the  Armament  Division  at  Eglin  AFB,  FLA  has  27,  the  Ballistic  Missile 
Organization  at  Norton  AFB,  California  has  17  and  the  Space  Division  at  Los 
Angeles  has  16.  Both  Ballistic  and  Space  activities  regularly  supplement  their 
work  force  with  consultants.  Aerospace  and  TRW  respectively. 


3rd.  Our  buying  divisions  use  matrix  or  functional  management  in  manufacturing 
as  deemed  appropriate  by  their  Commanders. 


4th.  Our  PESO  people  are  located  at  and  under  the  management  of  our  buying 
divisions.  They  are  used  to  conduct  any  of  the  reviews  and  assessments  which 
make  up  the  Air  Force  Manufacturing  Review  Program,  which  will  be  presented  to 
you  in  a  few  minutes  by  my  sidekick,  Dan  Murphy. 


5th.  The  assessments  conducted  by  our  PESO  people  and  other  manufacturing 
staff  in  the  Buying  Divisions  are  considered  as  being  independent  manufacturing 
assessments  as  they  are  not  conducted  under  the  supervision  or  control  of  the 
program  managers. 
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6th.  The  independent  manufacturing  assessments  we  conduct  at  Headquarters 
AFSC  place  heavy  emphasis  on  very  early  and  very  complete  planning.  In  this 
regard  we  require  the  buying  divisions  to  submit  PRR  plans  to  us  within  90 
days  after  award  of  FSED.  With  this  emphasis  on. early  and  complete  planning 
we  can  feel  safe  in  completing  our  independent  manufacturing  assessments  with 
minimum  followup,  observation  and/or  participatiQn  in  the  PRR. 

7th.  The  Air  Force  has  been  conducting  PRRs  for  over  ten  years  compared  to 
just  a  few  years  for  the  other  services.  These  have  been  conducted  in 
accordance  with  AFSCR84-2,  since  its  issuance  in  November  1971.  It  is  very 
similar  and  the  forerunner  of  DODI  5000.38,  which  was  published  in  1979. 

Despite  our  long  and  extensive  experience  it  is  clear  from  all  of  the  presenta¬ 
tions  yesterday  that  we  can  learn  a  lot  from  the  Army,  Navy,  DCAS  and  DOD/PESO 
experience . 

8th.  And  lastly,  we  have  a  unique  way  of  incentivizing  our  PRR  enthusiasts 
to  keep  them  from  becoming  over  zealous  and  exceeding  their  allocated  presenta¬ 
tion  time.  We  use  my  wife’s  timer,  which  I  brought  alcng  for  an  earlier  meeting 
this  week,  and  make  it  clear  that  their  OER  and  Merit  Pay  is  in  jeopardy  if  the 
bell  goes  off  -  -  and  they  may  as  well  hang  it  up  if  they  go  on  for  over  five 
minutes . 

Thank  you  for  your  attention  and  at  this  time  it  is  my  pleasure  to  call  on  Dan 
Murphy,  who  is  the  other  half  of  the  AFSC  Huntley,  Brinkley,  to  double  time  up 
here  and  put  PRRs  in  focus  with  other  very  important  manufacturing  reviews  that 
are  included  in  our  Manufacturing  Review  Program. 


PRR  CONFERENCE 
PREAMBLE 

TO  DECIDE  OR  NOT  TO  DECIDE,  THAT  IS  THE  QUESTION.  WHETHER  'TIS.  .  . 
AND  BY  OPPOSING  END  THEM.  'TIS  A  CONSUMMATION  DEVOUTLY  TO  EE 
WISHED— AYE  THERE'S  THE  RUB— FOR  THERE  AP.E  MILLIONS  OF  US  IN  THE 
VALLEY  OF  DECISION. 

AS  HUMAN  BEINGS,  OUR  LEGACY  IS  TO  MAKE  DECISIONS  AND  WE  MAKE  THEM 
EVERY  WAKING  HOUR.  THEY  RANGE  THE  SPECTRUM  FROM  THE  TINIEST  a 
INCONSEQUENTIAL  TO  THE  MOMENTOUS  a  TITANIC,  BUT  MAKE  THEM  WE  WILL. 

VIE  MAY  DECIDE  NOT  TO  DECIDE — BUT  DECIDE  HE  WILL. 

WE  ALL  FALL  HEIR  TO  THE  TINY  DECISION  (SHALL  WE— LUNCH  AT  CLUB- 
PLAY  BINGO  TONIGHT)  AND  A  FEW  MAKE  DECISIONS  THAT  ARE  TRULY 
OVERAWING.  WE,  IN  THE  ACQUISITION  BUSINESS,  GET  SOME  THAT  AFFECT 
NOT  ONLY  THE  WELL-BEING  OF  OUR  COUNTRY,  NOT  ONLY  THE  W.  HEMISPHERE, 

S  NOT  ONLY  THE  WORLD,  BUT  SPACE  a  THE  UNIVERSE.  WE  ARE  TRULY  ON 
THE  LIP  OF  THE  NUCLEAR  a  SPACE  AGE  a  OUR  DECISIONS  ARE  FORERUNNERS— 
BUILDING  BLOCKS  OF  THE  DECISION  PROCESS-AFSARC  a  DSARC . 

IN  MAKING  THESE  CATACLYSMIC  DECISIONS,  THERE'S  ONLY  ONE  WHOLELY 
WRONG  DECISION  a  THAT'S  TO  "DECIDE  NOT  TO  DECIDE,"  LET'S  SEE  HOW 
THE  AIR  FORCE  PROCESS  CONTRIBUTES  a  LOOK  AT  WHAT  WE'RE  GOING  TO 
CONSIDER. 
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AFSC  OPENING  REMARKS 


AF  MANUFACTURING  REVIEW  PROGRAM 

Since  the  larger  costs  of  an  acquisition  program  normally  occur  during  the 
production  phase,  the  AF  has  developed  a  series  of  fifteen  formal  reviews  to 
reduce  risks  in  the  manufacturing  area. 

ESSENCE  -  A  planned  series  of  interrelated  manufacturing  reviews  constitutes  a 
manufacturing  review  program.  These  reviews  provide  opportunities  to  put 
emphasis  on  producibil ity  early  in  the  acquisition  cycle,  to  integrate  manufacturing 
considerations  throughout  the  life  of  the  program,  and  assess  status  of  planning 
for  the  production  phase. 

REVIEWS 

Some  of  the  reviews  are  performed  by  the  government  and  some  by  contractors. 

Some  of  the  reviews  are  used  to  evaluate  contractors  without  regard  to  any 
specific  program.  Some  are  performed  before  a  particular  program  contract  is 
awarded  and  others  are  conducted  during  the  performance  of  a  given  contract. 

All  reviews,  however,  evaluate  a  contractor's  current  or  potential  capability 
and  performance,  identify  manufacturing  risks  and  assess  ability  to  transition 
from  one  phase  to  the  next  succeeding  one  in  the  program's  life  cycle. 

KEY  REVIEWS  The  key  reviews  in  a  Manufacturing  Review  Program  are  the 
Manufacturing  Management/Production  Capability  Review  (MM/PCR)  and  the 
Production  Readiness  Review(PRR). 

MM/PCR 

PURPOSE  -  MM/PCR  is  an  evaluation  (before  the  fact,  that  is,  before  award  of 
contract)  of  competing  manufacturer's  planned  manufacturing  management  system 
and  production  capability  and  capacity  to  meet  known  production  requirements 
of  proposed  systems  considering  firm  and  projected  business.  Its  primary 
purpose  is  to  support  the  source  selection  process  and  is  conducted  by  the  SPO. 

A  pre-award  survey  is  requested  by  the  SPO  and  conducted  by  the  Plant  Cognizant 

✓ 

Activity. 
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WHEN  -  The  MM/PCR  is  an  investigation  conducted  by  a  Program  Office  manufacturing 
function  during  source  selection  process  to  ascertain  how  well  prospective 
contractors  can  accomplish  the  manufacturing  tasks  of  a  proposed  program 
throughout  its  entire  life  cycle.  The  MM/PCR  can  be  performed  during  the 
demonstration  and  validation  phase  and  the  full  scale  engineering  development 
phase  for  selection  of  validation  contractors  and  selection  of  FSED  contractors. 

USED  -  Selection  of  Contractors  -  The  MM/PCR  is  conducted  at  prospective 
contractors  plants  during  the  Source  Selection  Process.  It  is  an  ideal  basis  for 
comparison  of  contractors  and  it  aids  and  influences  the  decision  and  selection 
process  during  validation  and  FSED  phases.  The  MM/PCR  data  acquired  at  the 
winning  contractor's  facility  is  the  departure  point  (reference)  for  the 
subsequent  PRR  program.  This  data  should  also  be  used  by  the  Should  Cost  Team 
and  should  enable  them  to  more  quickly  focus  attention  on  high  risk  design  areas. 
The  data  should  also  be  used  by  the  Source  Selection  Board  (SSB)  to  assist 
them  in  their  evaluation.  The  data  from  the  MM/PCR,  conducted  on  the  selected 
contractor,  provides  a  Traceability  base  line  for  the  subsequent  PRRs.  This 
comparison  of  the  baseline  data  to  later  reviews  reveals  trends  and  provides 
insight  to  the  contractor's  progress  and  capabilities. 

PRR 

PURPOSE  -  The  PRR  is  a  formal  inspection  conducted  by  the  Program  Office  (after 
the  fact)  in  support  of  MILESTONE  III  decision.  It  is  to  identify  remaining 
manufacturing  risks  by  determining  whether  or  not: 

a.  Systems  are  ready  for  efficient  and  economical  production. 

b.  The  Production  engineering  problems  have  been  resolved. 

c.  The  contractor  has  performed  adequate  planning  for  production. 

WHEN  -  The  PRRs  are  conducted  by  the  program  office  and  during  the  Full  Scale 
Engineering  Development  (FSED)  in  support  of  MILESTONE  III  decisions. 
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PRR  PROGRAM 


DEFINE  -  There  seems  to  be  an  implicit  idea  that  all  facets  of  a  PRR  are 
covered  in  one  evaulation.  In  the  incremental,  the  full  scope  of  the  PRR  is 
broken  down  and  different  element  or  segments  are  evaluated  at  different  times 
during  the  FSED.  So  incremental  PRRs  are  really  elements  or  segments  of  a 
complete  PRR.  This  is  particularly  true,  because  in  the  period  immediately 
following  the  FSED,  some  of  the  production  engineering  efforts  have  just  begun 
or  are  beginning.  Consequently,  all  that  can  be  examined  is  the  treatment  and 
efforts  accorded  to  the  plans  and  programs  identified  in  the  MM/PCR.  There 
are,  however,  features  of  any  major  defense  aerospace  program  that  must  be 
treated  or  attended  to  early  i.e.;  long  term  lead;  critical  materials, 
facilities  and  labor  skills.  Once  these  strategic  items  have  been  addressed 
in  their  strategic  time  frame,  only  periodic  surveillance  and  traceability 
of  their  trends  are  required. 

ADVANTAGES  -  Incremental  PRRs  provide  the  following  advantages: 

Many  Locations  -  In  large  systems  with  many  contractors  and  subcontractors 
incremental  Reviews  are  more  practical  for  reviewing  all  the  plants  involved. 
Incremental  PRRs  more  readily  adapt  themselves  to  covering  many  facilities. 

Time  Constraints  -  Strategic  elements  for  consideration  must  be  considered 
at  appropriate  times  during  the  FSED  phase  in  order  to  be  effective.  This 
allows  items  requiring  extended  time  to  be  considered  early. 

Team  Size  -  The  incremental  PRR  allows  smaller  team  sizes  since  fewer 
segments  are  reviewed.  In  addition,  the  teams  can  analyze  the  elements  to  a 
greater  depth.  Smaller  teams  cause  less  contractor  disruption. 

Traceability  and  Reference  -  The  PRR  iterations  provide  a  readily  identifiable 
traceability  of  key  production  related  items.  Data  from  initial  PRR  iterations 
also  provide  an  excellent  reference  on  where  to  focus  at  subsequent  Production 
Reviews  and  suggest  other  areas  to  investigate  and  provide  a  timely  view  of 
trends  and  the  contractor's  progress. 
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PRR  PLANS 


PRR  plans  have  been  neglected  and  need  emphasis.  They  should  be  tailored  to 
the  system--not  generalities.  They  should  contain  as  a  minimum  a  description 
of  the  system  and  the  principal  contractors  involved.  It  should  address  for 
the  first  iteration  identification  of  critical  materials  and  long  lead  times, 
advance  funding,  facilities  and  equipment  and  availability  of  labor  skills. 

HQ  LESSONS  LEARNED 

Unique  Requirements  -  involves  NATO  involvement;  Rationalization  Standardization 
and  Interoperability  (RSI)  FSED  strategy  should  be  identified  which  would  lead 
to  addressing  concurrency,  affordi bil ity  and  other  strategic  items  at  appropriate 
times.  The  following  items  highlight  other  principal  lessons  learned: 

Manufacturing  Involvement  -  IG  cited  -  Basis  for  PRR  Planning. 

Procurement  Plans  -  Program  Management  Plans  -  SOW  preparation  -  Cost  Proposal 
Analysis  -  Component  Breakout  -  GFAE  Management  -  Subcontractor  Management. 

Long  Lead  Items  and  Critical  Materials  should  be  identified  and  actions 
initiated  for  applicable  priorities. 

Manufacturing  Technology  contracts  held  by  contractors  should  be  identified 
and  areas  where  they  are  implemented.  These  are  candidates  for  review  early 
and  at  successive  iterations. 

Labor  Features  -  Skills  availability  and  potential  labor  problems  should 
be  identified. 

EVALUATION 

These  are  question  areas  that  have  arisen  at  HQ.  We  solicit  your  discussion 
and  contributions.  Some  of  these  are: 

EMPHASIS  -  is  looked  at 

MM/PCR  -  Is  there  enough  emphasis? 

Source  Selection  -  Is  there  adequate  criteria?  Is  it  enough? 
Manufacturing  Modernization  &  Productivity  -  Have  we  done  enough  there? 
Especially  in  view  of  world  market? 
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USE 


MM/PCR  -  Used  effectively? 

Key  Reviews  -  Are  they  done  in  an  integrated  fashion  or  functionally? 
PRR  Improvement  -  How  can  we  improve  the  PRR  process? 


SUMMARY 

Remember  -  AF  manufacturing  review  program 
series  of  reviews 
Remember  -  Key  reviews 

MM/PCR  (before  fact) 

PRR  (after  the  fact) 

Remember  -  Decision  process  it  is  a  management  tool 
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EPILOGUE 


SUMMARY 

So  IN  SUMMARY  LET'S: 

REMEMBER 

That  the  Air  Force  Manufacturing  Review^Erpsbm  is  a  series 

OF  REVIEWS  RELATING  TO  MANUFACTURING, 

REMEMBER 

That  the  Key  Reviews  are: 

*  MM/PCR  -  Manufacturing  Management  Production  Capability 

Rev i ew 

-  Looks  at  potential  before  -  fact 

*  PRR  -  Production  Readiness  Review 

-  Looks  at  current  Production  Readiness  -  after  fact 

REMEMBER 

*  These  reviews  are  building  blocks  of  the  decision 

PROCESS  &  ARE  BASIS  FOR  MEANINGFUL  DECISION  & 
ACQUISITION  MANAGEMENT 
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AIR  FORCE 


MANUFACTURING  REVIEW 

PROGRAM 


PRESENTOR:  D.T.  MURPHY 
HQ  AFSC/PMD 
19  NOV  80 


CONTENTS 

•  ESSENSE 

•  TYPE  REVIEWS 

•  KEY  REVIEWS 

•  PRR  PROGRAM 

•  PRR  PLANS 

•  HQ  LESSONS  LEARNED 

•  EVALUATION 
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ESSENCE 


•  SERIES  MANUFACTURING  REVIEWS 

•  EMPHASIS  PROOUCIBILITY-EARLY 

•  INTEGRATES  MANUFACTURING  CONSIDERATIONS 
THRU  PROGRAM  LIFE 

•  ASSESSES  PROOUCTION  PLANNING 


TYPES  OF  REVIEWS 

•  GOVERNMENT  PERFORMED 

•  CONTRACTOR  PERFODMEO 

•  EVALUATE 

•  BEFORE  SPECIFIC  AWARO 

•  OURING  CONTRACT  PERFORMANCE 


208 


KEY  REVIEWS 


•  MM/PCR 

•  MANUFACTURING  MANAGEMENT  PROOUCTION 
CAPABILITY  REVIEW 

•  PRR 


•  PRODUCTION  READINESS  REVIEW 


toe 


1«3  9 
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NEW  a  CAPABILITY 


MM/PCR 


•  PURPOSE  (BEFORE  FACT) 

•  EVALUATION  COMPETING  CONTRACTORS 

-  MANUFACTURING  MANAGEMENT  SYSTEM 

-  PRODUCTION  CAPABILITY  -  CAPACITY 
FIRM  &  PROJECTEO 

•  WHEN 

•  PHASES  -  DEMONSTRATION  &  VALIDAT iON/FSED 

•  USED: 

•  PRE-AWARD  SURVEY 

•  SELECTION 

•  REFERENCE  -  PRRs 


PRR 

•  PURPOSE  (AFTER  FACT) 

•  FORMAL  INSPECTION 

•  IDENTIFIES  RISKS 

-  SYSTEM  READY  -  OPTIMUM  PRODUCTION 

-  PRODUCTION  ENGINEERING  PROBLEMS  SOLVEO 

-  CONTRACTOR  PLANNING  AOEQUATE 

•  WHEN 

•  DURING  FSED 

•  SUPPORT  MILESTONE  III 
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PRR  PROGRAM 


•  DEFINE 

•  INCREMENTAL  REVIEWS 

•  ADVANTAGES 

•  MANY  LOCATIONS 

•  TIME  CONSTRAINTS 

-  STRATEGIC  ITEM 

•  SMALLER  TEAMS 

•  DEPTH  OF  ANALYSIS 

•  TRACEABILITY  -  REFERENCE 


HQs  LESSONS  LEARNED 

•  UNIQUE  REQUIREMENT 

•  NATO 

•  RSI 

•  AFFORDABILITY 

•  CONCURRENCY 

•  MANTECH  CONTRACTS 

•  LABOR  FEATURES 

•  PRR  PUNS 
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STRATEGIC  ISSUES  (TIME  CONSTRAINTS) 


PRR  PLANS 


•  NEGLECTED 

•  TAILORED  TO  SYSTEM 

•  SYSTEM  DESCRIPTION 

•  CONTRACTORS 

•  TIME  CONSTRAINTS  -  EARLY 

•  LONG  LEAD 

•  FUNDING 

•  FACILITIES 


SUMMARY 


•  AF  REVIEW  PROGRAM 

•  SERIES 

•  FEATURES 

•  BEFORE  (MM/PCR) 

•  AFTER  (PRR) 
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EVALUATION 

•  EMPHASIS  -  ENOUGH? 

•  MM/PCR 

•  SOURCE  SELECTION  CRITERIA 

•  MANUFACTURING  MODERNIZATION  &  PRODUCTIVITY 

•  USES  •  EFFECTIVE 

•  MM/PCR 

•  INTEGRATED  REVIEWS  •  FUNCTIONAL 

•  IMPROVE  PRRs  •  HOW 
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1980  DOD  PRODUCTION  READINESS  REVIEW  CONFERENCE 


SCRIPT  TO  ACCOMPANY 

"PRRs  from  the  Electronic  Systems  Division  Perspective" 

Presented  By 
Richard  A.  L'Heureux 


The  information  presented  here  describes  what  works  for  AF  Electronic  Systems 
Division  (ESD)  regarding  PRRs.  It  is  not  intended  to  make  a  judgement  of  other 
approaches  -  it  is  what  works  for  ESD  -  if  the  information  can  be  of  value  to  you  the 
presentation  will  have  been  worthwhile. 

Regarding  review  techniques,  approach  the  PRR  task  as  though  you  were  to  build  the 
hardware  -  be  a  doer  don't  listen  to  extensive  contractor  brochuremanship.  We  have  found 
that  the  evaluation  of  all  aspects  of  a  selected  item  of  hardware  is  preferable  to  a  broad 
brush  view  of  all  the  system  -  a  core  sample  tests  the  entirety  of  the  production 
planning/design  readiness  of  the  hardware. 

Go  on  record  in  the  Statement  of  Work  (SOW)  of  the  contract  stating  that  the 
government  will  not  ask  the  contractor  for  any  data  other  than  that  normally  used  in  the 
conduct  of  the  effort  -  but  state  further  that  the  contractor  will  make  that  existing 
information  available  for  government  review  and  provide  limited  assistance  to  find  and 
extract  the  data. 

Limit  the  team  size  and  duration  of  the  review(s)  -  it's  early  to  do  it  and  not  the 
easiest  task  but  if  we  don't  want  to  have  huge  bills  for  PRRs  we  have  to  do  it.  Use  outside 
help  in  specialized  areas  and  try  not  to  have  so  much  program  office  participation  that  it 
overpowers  the  team  at  the  expense  of  the  objectivity  of  the  review. 

All  of  the  foregoing  rolls  up  into  a  recommended  SOW  as  shown  here. 

The  PRR  has  a  clear  purpose  and  it  should  not  be  considered  a  substitute  for  or  an 
adjunct  to  any  of  the  other  reviews/surveys  we  accomplish  -  to  do  that  would  dilute  the 
value  of  both.  The  PRR,  however,  does  gather  data  from  all  the  sources  shown  here  to 
highlight  fruitful  areas  for  review. 

As  mentioned  before,  we  prefer  to  take  a  core  sample  of  hardware  and  analyze  all 
the  aspects  of  it  to  the  broad  brush  treatment.  Where  foreign  subcontractors  are  used,  we 
should  concentrate  on  insuring  that  there  has  been  an  adequate  transfer  of  total 
information  necessary  to  produce  and  to  insure  that  it  can  be  assimilated  into  the  sub¬ 
contractor's  system. 

To  give  an  example  of  what  we  feel  is  the  necessary  flexibility  of  the  PRR  here  is  a 
special  emphasis  subject  -  productivity. 

What  results  do  we  expect  from  a  PRR?  We'd  much  rather  see  two  or  three  well 
defined  and  supported  conclusions  with  recommendations  for  reducing  the  risk  than  a  sea 
of  general  observations  resulting  in  a  noncommittal  report.  Be  brief  in  the  reports  -  keep 
plenty  of  backup  data  -  but  have  a  heart  on  the  reader! 
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Implementation  -  get  the  support  of  the  affected  program  management  -  tell  them 
good  things  as  well  as  bad,  and  by  all  means  establish  a  followup  plan  -  with  assigned  OPRs 
—  then  follow  it  up  on  a  regular  schedule. 

Summary  -  have  the  foresight  to  plan  the  PRR  months  (even  years)  before  it  is  to  be 
done  -  in  the  contract.  Do  the  review  -  don't  go  and  listen  to  contractor  presentations. 
Support  your  conclusions  -  establish  a  remedial  plan  where  necessary  -  and  followup  on  the 
action  items!  *” 
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PRODUCTION  READINESS 
REVIEWS 
FROM  THE 

ELECTRONIC  SYSTEMS  DIVISION 
PERSPECTIVE 


OVERVIEW 


OBJECTIVES 
REVIEW  TECHNIQUES 
DATA  REQUIREMENTS 
TEAM  SIZE  AND  COMPOSITION 
SAMPLE  STATEMENT  OF  WORK  ENTRY 
RELATIONSHIP  TO  OTHER  REVIEWS 
READINESS  REVIEW  TASKING 
SPECIAL  SUBJECT-PRODUCTIVITY 
EXPECTED  RESULTS  OF  PRRs 
IMPLEMENTATION  OF  FINDINGS 
SUMMARY 
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OBJECTIVES 


ASSESS  STABILITY  OF  DESIGN 

DETERMINE  ADEQUACY  OF  PRODUCTION  PLANNING 


REVIEWS  TECHNIQUES 

MANUFACTURING  DIRECTORATE  RESPONSIBILITY 

MODUS  OPERAND  I  -  BE  A  DOER;  NOT  A  LISTENER  TO  CONTRACTOR 
PRESENTATIONS 

LENGTH  -  TRY  TO  LIMIT  TO  5  DAYS  MAXIMUM  (THIS  SHOULD  BE 
IN  SOW) 

CORE  SAMPLE  VS  BROAD  BRUSH 

AREAS  TC  BE  REVIEWED  COORDINATED  WITH  PROGRAM  OFFICE 
INSURE  CAO  INVOLVEMENT 
ORGANIZE  -  SCHEDULE  -  COORDINATE 
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DATA  REQUIREMENTS 


DO  NOT  REQUIRE  DATA  OTHER  THAN  THAT  GENERATED  IN  THE 
COURSE  OF  THE  PROGRAM  -  PUT  THIS  IN  WRITING  (SOW) 

DO  REQUIRE  ACCESS  TO  CONTRACTOR  DATA  IN  SUPPORT  OF 
PRR  (SOW) 

DO  STATE  LIMITED  CONTRACTOR  ASSISTANCE  WILL  BE 
NECESSARY  IN  LOCATING/EXTRACTING  DATA 


TEAM  SIZE  AND  COMPOSITION 


PESO  INVOLVEMENT  AT  ESD 

LIMIT  TEAM  SIZE  AND  MAKE  THAT  A  MATTER  OF 
EARLY  RECORD  (THREE  TO  EIGHT) 

ADDRESS  MOST  SIGNIFICANT  AREAS  IDENTIFIED^ 
BY  MANUFACTURING  AND  COORDINATED  WITH 
PROGRAM  OFFICE 

CALL  ON  OUTSIDE  SUPPORT  (LABHRATOP IES/PESO/ 
NON-PROGRAM  DESIGN  ENGINEERS,  QUALITY 
ASSURANCE  EXPERTS/ETC) 

PROGRAM  OFFICE  REPRESENTATION 


219 


SAMPLE  STATEMENT  OF  WORK  ENTRY 


"THE  CONTRACTOR  SHALL  SUPPORT  THE  CONDUCT  OF  INCREMENTAL  PRODUCTION 
READINESS  REVIEWS  (PRRs)  DURING  THE  FSED  PROGRAM. 

"CONTRACTOR  SUPPORT  REQUIREMENTS  WILL  BE  LIMITED  TO  THAT  NECESSARY  TO 
IDENTIFY  SOURCES  OF  INFORMATION/DATA  TO  PRR  TEAM  MEMBERS  AND  TO  LIMITED 
PERSONAL  INTERVIEWS  DURING  THE  REVIEW  FOR  CLARIFICATION  0F  DATA  ASSESSED  BY 
TEAM  MEMBERS.  THERE  WILL  BE  NOT  MORE  THAN  THREE  REVIEWS  CONDUCTED,  TWO  OF 
WHICH  WILL  HAVE  NOT  MORE  THAN  FIVE  GOVERNMENT  REPRESENTATIVES  AND  WILL  NOT 
EXCEED  FIVE  DAYS  IN  THE  CONTRACTOR'S  FACILITY.  THE  THIRD  AND  FINAL  REVIEW  TO 
BE  CONDUCTED  NEAR  THE  END  OF  FSED  PHASE  WILL  BE  ACCOMPLISHED  BY  NOT  MORE 
THAN  SEVEN  GOVERNMENT  REPRESENTATIVES  AND  WILL  LAST  NOT  MORE  THAN  FIVE 
WORKDAYS  IN  THE  CONTRACTOR'S  FACILITY,  THE  CONTRACTOR  WILL  NOT  BE  TASKED 
TO  PREPARE  ANY  DATA  TO  SUPPORT  THE  PRRs  WITH  THE  EXCEPTION  OF  SLIDES  OR 
OTHER  INFORMATION  DESIRED  BY  THE  CONTRACTOR  TO  EXPEDITE  THE  REVIEW. 


RELATIONSHIP  TO  OTHER  REVIEWS 


A  PRR  USES  INFORMATION  FROM  ALL  OTHER  PROGRAM/ 
FUNCTIONAL  REVIEWS/SOURCES 

A  PRR  IS  NOT 

-  AN  MM/PC 

-  A  MFG  ASSESSMENT  REVIEW 

-  A  COST  REVIEW 

-  A  PDR  OR  CDR 

-  A  PMR 

-  A  PREAWARD  SURVEY 
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COST 


<s> 


READINESS  REVIEW  TASKING 


ESD  APPROACH  LEANING  TOWARD  INTENSIVE  LOOK  AT  A  SMALL 
SELECTED  AREA,  E.G.  ONE  SUBSYSTEM  TRACED  TO  COMPONENT 
PARTS,  SCHEDULES,  WORK  INSTRUCTIONS 

WHERE  MAJOR  SUBCONTRACTORS  ARE  INVOLVED,  EITHER  DO  A 
SEPARATE  PRR  (WITH  KNOWLEDGE  AND  CONSENT  OF  PRIME)  OR 
CLOSELY  MONITOR  PRIME'S  REVIEW  (IDENTIFY  IN  SOW) 


FOREIGN  SUBCONTRACTORS  -  EMPHASIS  ON  INFORMATION  TRANSFER 


REVIEW  AREA.  ASSIGNMENT  -  TRY  TO  LIMIT  TO  ONE  PEP  MEMBER  - 
CAO  ASSIGNED  WHERE  SYSTEM  FAMILIARITY  MOST  IMPORTANT 
(QA,  PLANT  LAYOUT,  ETC) 
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SPECIAL  SUBJECT  -  PRODUCTIVITY 


IF  REQUIRED  IN  CONTRACT,  THE  FOLLOWING  PRODUCTIVITY  ASSURANCE 
ELEMENTS  SHOULD  BE  ASSESSED  IN  PRR: 

-  HAS  THE  CONTRACTOR  IMPLEMENTED  PRODUCTIVITY 
ASSURANCE  PROGRAM  WHICH  FULLY  ADDRESSED  THE 
PROGRAM'S  NEEDS? 

-  HAS  THE  RESULTS  OF  THE  PRODUCTIVITY  ASSURANCE 
PROGRAM  BEEN  AGGRESSIVELY  PURSUED  IN  FULFILLMENT 
OF  PROGRAM  NEEDS? 

-  WILL  PRODUCTIVITY  ASSURANCE  IMPROVEMENTS  BE 
OPERABLE  IN  SUFFICIENT  TIME  TO  IMPACT  THE  SUBJECT 
PROGRAM? 


EXPECTED  RESULTS  OF  PRRs 


DEFINITIVE,  SUPPORTABLE  CONCLUSIONS  IN  SELECTED  REVIEW 
AREAS  RELATIVE  TO  READINESS  FOR  PRODUCTION  PHASE 

IDENTIFICATION  OF  AREAS  REQUIRING  ADDED  PROGRAM/ 
MANUFACTURING  MANAGEMENT  EMPHASIS  IN  PRODUCTION 

BRIEF  REPORTS  CONSISTENT  WITH  ENOUGH  RATIONALE  TO 
SUPPORT  REVIEW  FINDINGS  -  BRIFF 
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IMPLEMENTATION  OF  FINDINGS 


EXECUTIVE  LEVEL  BRIEFING  TO  PROGRAM  DIRECTOR 
AND  STAFF 

PRESENT  POSITIVE  AS  WELL  AS  NEGATIVE  REVIEW 
FINDINGS 

RECOMMEND  FOLLOW-UP  ACTIONS/SUGGEST  ACTION  AGENCIES 
(PROGRAM  OFF  I CE/CAO/CONTRACTOR) 

ESTABLISH  FOLLOW-UP  REVIEW  CYCLE  (WITH  REPORTING 
AS  APPROPRIATE)  TO  INSURE  THAT  FINDINGS  ARE  ADDRESSED 


SUMMARY 


PLANNING  -  PUT  RIGHT  REQUIREMENTS  IN  THE  CONTRACT/ 

BE  REALISTIC/BE  COST  EFFECTIVE/USE  PESO 
IN  ADVISORY/APPROVAL  ROLE 

EXECUTION  -  BE  A  DOER  NOT  A  LISTENER/CALL  ON  WHATEVER 
EXPERTISE  NECESSARY/STAY  ON  TRACK 

RESULTS  -  SUPPORTABLE  CONCLUSIONS/MAKE  IMPACT  FELT/ 
FOLLOW-UP 
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SLIDE  1  -  COMMAND  LOGO,  I  AM  MAJOR  STAN  ZALACE  OF  THE 
ARMAMENT  DIVISION'S  DIRECTORATE  OF  MANUFACTURING, 

SLIDE  2  -  MISSION  AND  PROGRAMS,  THIS  NEXT  SLIDE  SHOWS  OUR  PRI¬ 
MARY  MISSION  AND  LISTS  SOME  OF  THE  PROGRAMS  WE  ARE  WORKING  ON. 
OUR  MISSION  ENCOMPASSES  THE  ENTIRE  SPECTRUM  OF  ACTIVITIES 
FROM  RESEARCH,  TECHNOLOGY  AND  DEVELOPMENT  PLANNING  TO  INITIAL 
ACQUISITION  OF  ARMAMENT  FOR  THE  AIR  FORCE  INVENTORY,  OUR 
PROGRAMS  RANGE  IN  SIZE  AND  COMPLEXITY  FROM  20MM  AMMUNITION  AT 
ONE  END  OF  THE  SPECTRUM  TO  CONVERSION  OF  THE  F-100  SUPER  SABRE 
INTO  A  REMOTELY  PILOTED  FULL  SCALE  AERIAL  TARGET  AT  THE  OTHER 
END, 

SLIDE  3  -  AD  ORGANIZATION.  THIS  SLIDE  SHOWS  MY  ORGANIZATION 
WITHIN  THE  DEPUTATE  FOR  CONTRACTING  AND  MANUFACTURING  AND  ALSO 
THE  PROGRAM  OFFICE  STRUCTURE  UNDER  THE  DEPUTY  FOR  ARMAMENT 
SYSTEMS,  MANUFACTURING  IS  A  MATRIXED  ORGANIZATION;  THAT  IS, 

THE  MAJORITY  OF  OUR  PERSONNEL  ARE  COLOCATED  IN  THE  VARIOUS 
PROGRAM' OFF  ICES,  WHILE  MOST  OF  THE  MEMBERS  OF  A  PRODUCTION 
READINESS  REVIEW  TEAM  COME  FROM  THE  PROGRAM  OFFICE,  THE  TEAM 
DIRECTOR  IS  ONE  OF  THE  DIVISION  CHIEFS  FROM  THE  DIRECTORATE 
OF  MANUFACTURING,  WE  SHALL  SEE  LATER  THAT  THIS  IS  VERY 
SIGNIFICANT, 

SLIDE  4/5  -  HEAD  SCRATCHER/TITLE  SLIDE.  NOW  THAT  WE  HAVE 
SEEN  THE  OUTLINE  OF  THE  ORGANIZATIONS  PRIMARILY  INVOLVED 
IN  PRRS,  WE  CAN  LAUNCH  INTO  THE  MEAT  OF  MY  FIRST  BRIEFING. 
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IN  THE  PAST  THREE  YEARS,  THE  DIRECTORATE  HAS  PERFORMED  TWELVE 
PRRS.  THESE  RANGED  FROM  ONE  ON  AN  ELECTRONIC  BOMB  FUZE  TO 
ONES  ON  A  SUPERSONIC  BOMB  RACK  AND  AN  AIR  INFLATABLE  BOMB 
RETARDER, 

WE  HAVE  TWO  MORE  IN  PLANNING  AT  THIS  TIME,  TO  BE  PERFORMED  BY 
MARCH  1981.  MY  PURPOSE  HERE  TODAY  IS  TO  SHOW  YOU  HOW  WE 
CONDUCT  THESE  REVIEWS. 

SLIDE  6  -  OVERVIEW.  IN  ADDITION  TO  THESE  POINTS,  PLEASE  FEEL 
FREE  TO  ASK  QUESTIONS  AT  THE  END  OF  MY  PRESENTATION. 

SLIDE  7  -  REGULATORY  GUIDANCE. 

-  MOVING  INTO  THE  REASONS  FOR  THE  PRR,  WE  FIND  THAT  AD  WORKS 
WITHIN  THE  CONSTRAINTS  OF  THESE  DIRECTIVES.  ADDRESSING  THE  LOWER 
TWO  BULLETS: 

-  AFSCR  84-2  STATES  "THE  PRR  IS  A  FORMAL  INSPECTION..."  IT  ALSO 
SAYS  THAT  "THE  PRR  APPLIES  TO  COMPETITIVE  PROCUREMENTS." 

-  AD  SUP  1  TO  AFSCR  84-2  ADDS  "FOR  THOSE  PROGRAMS  IN  WHICH 

THE  FULL  SCALE  DEVELOPMENT  CONTRACTOR  IS  ANTICIPATED  TO  PERFORM 
THE  INITIAL  PRODUCTION  (THAT  IS,  SOLE  SOURCE  PROCUREMENT),  A 
FULL  PRR  WILL  BE  CONDUCTED  AT  THE  COMPLETION  OF  THE  FULL  SCALE 
DEVELOPMENT  PHASE." 

SLIDE  8  -  OBJECTIVE. 

-  THE  PRR  IS  A  FORMAL  INSPECTION  TO  DETERMINE  THREE  THINGS: 

•  IF  A  SYSTEM  OR  EQUIPMENT  DESIGN  UNDER  DEVELOPMENT  IS  READY 
FOR  EFFICIENT  AND  ECONOMICAL  QUANTITY  PRODUCTION 

•  THAT  ALL  IMPORTANT  PRODUCTION  ENGINEERING  PROBLEMS 
ENCOUNTERED  DURING  DEVELOPMENT  HAVE  BEEN  RESOLVED. 
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•  THAT  THE  CONTRACTOR  HAS  ACCOMPLISHED  ADEQUATE  PLANNING 
FOR  THE  PRODUCTION  PHASE. 

SLIDE  9  -  IMPORTANCE. 

»  ADDRESSING  THE  SECOND  BULLET  ON  THIS  SLIDE,  THE  QUESTION 
BEING  ASKED  IS:  FROM  A  PRODUCIBILITY  STANDPOINT,  IS  THIS 
DESIGN  READY  FOR  PRODUCTION? 

t  THE  THIRD  BULLET  IS  ASKING:  FROM  A  PRODUCTION  STANDPOINT, 
IS  THIS  COMPANY  READY  FOR  PRODUCTION? 

SLIDE  10,  11  -  OUTFITTING  A  TEAM  AND  THE  PRR  PLAN 

—  PRIOR  TO  LEAVING  FOR  A  CONTRACTOR'S  FACILITY  TO  PERFORM  A 
PRR,  THE  DIRECTORATE  OF  MANUFACTURING  WILL  HAVE  BUILT  A 
FORMAL,  DETAILED,  PLAN  FOR  CONDUCTING  THE  INSPECTION.  THIS 
PLAN  WILL  COVER  EVERY  ASPECT  OF  THE  PRR,  INCLUDING  THE  ACTUAL 
QUESTIONS  WHICH  WILL  EE  ASKED.  IN  OTHER  WORDS,  IT  IS 
CAREFULLY  TAILORED  FOR  THE  OCCASION. 

—  A  SHORT  PROGRAM  BACKGROUND  IS  INCLUDED  IN  THE  PLAN  TO 
ACQUAINT  ALL  TEAM  MEMBERS  WITH  THE  PROGRAM'S  HISTORY  AND 
CURRENT  STATUS.  THIS  IS  IMPORTANT  AS  THE  TEAM  IS  COMMONLY 
AUGMENTED  BY  PEOPLE  FROM  OUTSIDE  THE  ARMAMENT  DIVISION. 

--  THE  PLAN  LISTS  THE  TEAM  MEMBERS  AND  DEFINES  THE  RESPONSI¬ 
BILITIES  OF  EACH,  ESPECIALLY  PANEL  CHAIRMEN. 

(SLIDE  12  -  PRR  TEAM  INSTRUCTIONS) 

—  THE  PLAN  GIVES  A  DETAILED  SET  OF  INSTRUCTIONS  FOR  THE 
USE  OF  EACH  TEAM  MEMBER.  THESE  INCLUDE: 
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(SLIDE  13  -  QUESTION  SHEET) 

•  ANSWERING  AS  MANY  PRR  QUESTIONS  AS  POSSIBLE  AND  DRAFTING 
AREA  OF  CONCERN  SHEETS  PRIOR  TO  GOING  TO  THE  CONTRACTORS 
FACILITY. 

(SLIDE  14  -  AOC  SHEET)  (THOSE  QUESTIONS  WHICH  GENERATE  A  CONCERN 
OR  REQUIREMENT  FOR  ACTION  BY  THE  CONTRACTOR,  GOVERNMENT,  OR 
BOTH,  WILL  GO  ON  AOC  SHEETS) 

•  DEVELOPING  A  STATEMENT  OF  THE  TASKS  EACH  TEAM  MEMBER 
EXPECTS  TO  ACCOMPLISH  IN  SUPPORTING  THE  PRR. 

•  AT  THE  END  OF  THE  PRR,  DEVELOPING  A  NARRATIVE  SUMMARY 
FORMALIZING  THE  INDIVIDUAL'S  OBSERVATIONS  ABOUT  THE  CONTRACTOR'S 
PERFORMANCE,  THIS  SUMMARY  MUST  INCLUDE  AN  ASSESSMENT  OF  RISK. 

THE  SUMMARIES  ARE  GIVEN  TO  THE  PANEL  CHAIRMAN. 

•  IN  ADDITION  TO  THESE  THREE  ITEMS  JUST  COVERED,  PANEL 
CHAIRMEN  MUST,  WITH  HELP  FROM  PANEL  MEMBERS,  DEVELOP  THE 
CRITERIA  WHICH  THE  CONTRACTOR  MUST  MEET  TO  ACHIEVE  A  SUCCESSFUL 
PRR,  AND  DEVELOP  THE  LIST  OF  QUESTIONS  USED  TO  DETERMINE 
WHETHER  OR  NOT  THE  CRITERIA  HAVE  BEEN  MET. 

•  THE  PANEL  CHAIRMAN  WILL  SUMMARIZE  THE.  FINDINGS  OF  PANEL 
MEMBERS  AND  ASSIST  THE  TEAM  DIRECTOR  IN  WRITING  THE  FINAL 
PRR  REPORT. 

—  THE  PLAN  ALSO  PROVIDES  A  DETAILED  AGENDA  OF  ALL  EVENTS 
WHICH  WILL  TAKE  PLACE  AT  THE  CONTRACTOR'S  FACILITY. 

(SLIDE  15  -  POST  PRR  TASKS) 

—  POST  PRR  TASKS  INCLUDE: 

•  ASSESS  PRODUCTION  RISKS  AND  DEVELOP  ALTERNATIVES  TO  REDUCE 
OR  ELIMINATE  RISKS. 


228 


•  INITIATE  ANY  FORMAL  ACTION  TO  BE  ACCOMPLISHED  BY  THE 
COMPANY. 

•  FOLLOW-UP  TO  INSURE  ALL  CORRECTIVE  ACTIONS  ARE  COMPLETED. 

•  PREPARE  FINDINGS  FOR  PRESENTATION  TO  THE  AD  ACQUISITION 
REVIEW  COUNCIL. 

-  THE  PRR  PLAN  GOES  THROUGH  A  FORMAL  APPROVAL  CYCLE.  THE 
PLAN  IS  BUILT  WITHIN  THE  DIRECTORATE  OF  MANUFACTURING  AND 
SIGNED  BY  THE  TEAM  DIRECTOR  WHEN  COMPLETED.  THE  PROGRAM 
MANAGER  THEN  COUNTERSIGNS  THE  PLAN  TO  INDICATE  THAT  THE 
PROGRAM  OFFICE  CONCURS  WITH  IT.  THE  DIRECTOR  OF  MANUFACTURING 
THEN  REVIEWS  THE  FINALIZED  PLAN  AND  APPROVES  IT. 

-  TO  STREAMLINE  THE  MECHANICS  OF  MAKING  UP  THE  QUESTIONS  WHICH 
WILL  BE  ASKED;  WE  HAVE  BUILT  A  CATALOGUE  OF  QUESTIONS  ON  THE 
VARIOUS  AREAS  TO  BE  REVIEWED.  THIS  CATALOGUE  IS  BEING 
AUTOMATED  SO  THAT  IT  CAN  BE  QUICKLY  UPDATED;  AND  QUESTIONS 
SELECTED  FROM  IT  AND  PRINTED  OUT  AS  IS  OR  MODIFIED. 

SLIDE  16  -  OUR  TEAM 

SLIDE  17  -  THE  PRR  TEAM 

-  THIS  SLIDE  SHOWS  THE  STANDARD  AD  PRR  TEAM  CONFIGURATION. 

THE  USUAL  TEAM  SIZE  IS  8-10  PEOPLE;  AND  HAS  EXPANDED  TO 
14  WITH  AUGMENTEES.  (THE  PANELS  USUALLY  AUGMENTED  ARE 
PROGRAM  MANAGEMENT;  QUALITY  ASSURANCE;  AND  MANUFACTURING). 

WE  HAVE  ALSO  PERFORMED  A  PRR  WITH  ONLY  2  PEOPLE.  OUR  TEAM 
APPROACH  ALLOWS  FOR  A  THOROUGH  EVALUATION  OF  THE  CONTRACTOR'S 
CAPABILITY  AND  READINESS  TO  ENTER  PRODUCTION.  THE  DIRECTORATE 
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OF  MANUFACTURING  IS  A  MATRIXED  ORGANIZATION.  BECAUSE  OF  THIS, 

WE  HAVE  CALLED  UPON  OUR  PEOPLE  TO  SUPPORT  PRRS  BEING  CONDUCTED' 

FOR  OTHER  PROGRAM  OFFICES  THAN  THE  ONE  TO  WHICH  THEY  ARE  ASSIGNED. 
THE  BENEFITS  FROM  THIS  PRACTICE  ARE  THAT  PRODUCTION  PROBLEMS 
ARE  SEEN  FROM  A  NEW  VIEWPOINT,  WE  HAVE  HAD  A  CROSSFEED  OF 
IDEAS  BETWEEN  OUR  VARIOUS  BRANCHES,  AND  WE  HAVE  BEEN  ABLE  TO 
ESTABLISH  A  DEGREE  OF  PRR  EXPERTISE  IN  ALL  OF  OUR  PEOPLE. 

THE  I EAM  COORDINATOR  FUNCTIONS  AS  THE  MANUFACTURING  PANEL 
CHAIRMAN  AND  IS  PREPARED  TO  STAND  IN  FOR  THE  TEAM  DIRECTOR. 

-  ALL  OF  THE  PANELS  MAY  BE  AUGMENTED  BY  AFPRO,  NAVY,  ARMY, 

OR  DLA  PERSONNEL,  AS  THESE  AGENCIES  DESIRE.  IN  ADDITION, 

WE  MAY  AUGMENT  THE  MANUFACTURING  PANEL  WITH  A  MANUFACTURING 
OPERATIONS  CONSULTANT. 

SLIDE  13  -  MR.  JACK  R.  FRANKS  IS  AD's  MANUFACTURING  OPERATIONS 
CONSULTANT.  HE  ALSO  WORKS  IN  THES  SAME  CAPACITY  FOR  THE  F ~16 
PROGRAM  OFFICE. 

•  HE  SERVED  AS  THE  DIRECTOR  OF  MANUFACTURING  OPERATIONS  FOR 

THE  MCDONNELL-DOUGLAS  ASTRONAUTICS  CORPORATION  UNTIL  HIS  RETIRE¬ 
MENT. 

•  HE  IS  A  NTA IONALLY  KNOWN  EXPERT, 

•  HE  HAS  WORKED  ON  PROGRAMS  SUCH  AS  THE  AIM-9  MISSILE,  FMU-112 

FUZE,  bsu-49/b  air  inflatable  retarder,  and  amraam  missile 

FOR  US. 

SLIDE  19  -  CONDUCT  OF  THE  PRR 

“  WHEN  THE  TARGET  DATE  FOR  THE  PRR  IS  SET,  ALL  AFFECTED 

GOVERNMENT  AGENCIES  ARE  NOTIFIED  AND  AN  OUTLINE  OF  WHAT  IS 
INTENDED  PRESENTED  TO  THEM. 
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(SLIDE  20  -  SUPPORTING  CAST  SLIDE) 

THEY  ARE  INVITED  TO  AUGMENT  THE  TEAM  WITH  THEIR  OWN  PERSONNEL. 

IN  SOME  INSTANCES  WE  MAY  REQUEST  THAT  THEY  AUGMENT  THE  TEAM. 

-  AT  THIS  SAME  T l ME,  THE  CONTRACTOR  IS  NOTIFIED  OF  THE  INSPECTION 
AND  REPRESENTATIVE  FACTORS  TO  BE  ADDRESSED  BY  EACH  PANEL.  HE 

IS  NOT  GIVEN  THE  CRITERIA  OR  QUESTIONS.  HE  IS  ALSO  PROVIDED  A 
LIST  OF  ALL  AD  PERSONNEL  WHO  WILL  BE  VISITING  HIS  FACILITY. 

-  TWO  INITIAL  BRIEFINGS  ARE  GIVEN: 

e  THE  FIRST  IS  A  PRIVATE  BRIEFING  FOR  ALL  TEAM  MEMBERS 
IMMEDIATELY  PRIOR  TO  MEETING  THE  CONTRACTOR.  THIS  BRIEFING  COVERS 
SUCH  THINGS  AS  THE  AD  STRAIGHT  ARROW  PROGRAM  (AFR  30-30), 

GOVERNING  PERSONAL  CONDUCT  WHILE  AT  THE  CONTRACTOR'S  FACILITY; 
INTRODUCTION  OF  ALL  PERSONNEL  FROM  OTHER  AGENCIES;  FINAL  INSTRUC¬ 
TIONS;  AND  ANSWERING  ANY  QUESTIONS. 

•  THE  SECOND  BRIEFING  IS  TO  THE  CONTRACTOR.  IT  INTRODUCES 
THE  GOVERNMENT  TEAM,  EXPLAINS  WHAT  A  PRR  IS,  AND  ETC. 

-  ALL  TEAM  MEMBERS  MEET  FOR  A  DAILY  REVIEW  AFTER  FINISHING 
WITH  THE  CONTRACTOR  FOR  THE  DAY.  THIS  MEETING  UPDATES  STATUS, 
HIGHLIGHTS  AREAS  OF  CONCERN,  AND  PREPARES  FOR  THE  NEXT  DAY. 

-  WE  TRY  TO  ALWAYS  HAVE  THE  PCO  AS  A  MEMBER  OF  THE  PRR  TEAM, 

THE  PCO  SERVES  AS  THE  PROCUREMENT/SUBCONTRACTING  PANEL  CHAIRMAN. 

IT  IS  STRESSED  TO  BOTH  THE  CONTRACTOR  AND  TEAM  THAT  THE  PRR  TEAM 
MEMBERS  SHALL  IN  NO  WAY  GIVE  DIRECTION  TO  THE  CONTRACTOR. 

THE  PCO  IS  THE  ONLY  PERSON  AUTHORIZED  TO  ISSUE  DIRECTION.  HE 
IS  A  TEAM  MEMBER  TO  SEE  FIRSTHAND  THE  CONTRACTOR'S  STATUS, 

AND  HEAR  DIRECTLY  ANY  QUESTIONS  OR  PROBLEMS. 
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-  THE  FINAL  BRIEFING  TO  THE  CONTRACTOR  TELLS  HIM  WHETHER 
OR  NOT  HE  HAD  A  SUCCESSFUL  PRR,  AND  ANSWERS  HIS  QUESTIONS. 

SLIDE  21  -  REPORT  AND  BRIEFINGS 

-  THE  FINAL  REPORT  SUMMARIZES  THE  ENTIRE  PRR,  AND  IDENTIFIES 
AREAS  WHICH  WILL  REQUIRE  FOLLOW-UP.  IT  IS  CONSTRUCTED  IN  SUCH 
A  WAY  AS  TO  PROVIDE  TRACEABILITY  FROM  TOP  LEVEL  FINDINGS  DOWN 
TO  THE  SPECIFIC  FINDINGS  NOTED  BY  INDIVIDUAL  TEAM  MEMBERS. 

(SLIDE  22  -  AD  organization) 

-  THE  DEPUTIES  AND  THEIR  STAFFS  ARE  BRIEFED  IN  DETAL. 

-  ANY  FOLLOW-UP  VISIT  GENERATES  A  SEPARATE  REPORT  AND  BRIEFINGS. 
THE  FINAL  REPORT  PLUS  ANY  FOLLOW-UPS  IS  BRIEFED  TO  THE  AD/ARC 

TO  ASSIST  THEM  IN  MAKING  THEIR  PRODUCTION  RECOMMENDATION. 

SLIDE  23  -  INDEPENDENT  BRIEFINGS 

-  A  LESSON  WE  HAVE  LEARNED  IS  THAT  THIS  BRIEFING  APPROACH  IS 
CRITICALLY  IMPORTANT.  IF  THE  PRR  FINDINGS  WERE  ONLY  BRIEFED 
TO  THE  PROGRAM  OFFICE  THERE  IS  A  GOOD  CHANCE  THAT  BECAUSE  OF 
THE  MOMENTUM  OF  THE  PROGRAM  AND  PRESSURES  ON  THE  PROGRAM 
MANAGER  TO  PRESS  ON,  THE  FINDINGS  WOULD  BE  BURIED  AND  FORGOTTEN. 
THE  SEPARATE  BRIEFINGS  TO  THE  DEPUTY  FOR  CONTRACTING  AND 
MANUFACTURING  AND  TO  THE  AD  EXECUTIVE  COUNCIL  PREVENT  THIS. 
CONDUCTING  THE  PRR  INDEPENDENTLY  OF  PROGRAM  OFFICE  CONTROL  AND 
BRIEFING  THE  FINDINGS  AS  WE  DO  MAKES  THIS  INSPECTION  A  VERY 
VALUABLE  TOOL  FOR  DECISION  MAKING. 

SLIDE  24  -  PRR  FLOW  DIAGRAM 

THIS  FINAL  SLIDE  IS  A  FLOW  DIAGRAM  OF  HOW  WE  CONDUCT  PRRS 

AT  AD.  PLEASE  USE  IT  AS  A  REFERENCE  IN  FORMULATING  ANY  QUESTIONS 

YOU  MAY  HAVE  FOR  ME. 
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HOW  PRODUCTION  READINESS 
REVIEWS  ARE  CONDUCTED  BY 
THE  ARMAMENT  DIVISION 


MAJOR  STANLEY  ZALACE 
DIRECTORATE  OF  MANUFACTURING 
DEPUTATE  FOR  CONTRACTING  &  MANUFACTURING 
ARMAMENT  OIVISION 


MAJOR  STANLEY  ZALACE 
DIRECTORATE  OF  MANUFACTURING 
DEPUTATE  FOR  CONTRACTING  &  MANUFACTURING 
ARMAMENT  OIVISION 
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THE  ARMAMENT  DIVISION'S  PRIMARY  MISSION  IS  TO  DEVELOP.  TEST.  ANO  INITIALLY 
ACQUIRE  ALL  NONNUCLEAR  AIR  ARMAMENT  FOR  THE  AIR  FORCE'S  TACTICAL  ANO 

STRflTPRir  cnnrcQ 


ARMAMENT  DIVISION  ORGANIZATION 
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OVERVIEW 

•  REASONS  FOR  THE  PRR 

•  THE  PRR  PLAN 

•  THE  PRR  TEAM 

•  CONDUCT  OF  THE  PRR 

•  REPORT  AND  BRIEFINGS 
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REGULATORY  GUIDANCE 


•  DOD  DIRECTIVE  5000.38  (24  JAN  79)  PRR 

•  AFR  800-9  (3  JUL  78)  PRODUCTION  MANAGEMENT 
IN  THE  ACQUISITION  CYCLE 

•  AFSCR  84-2  (23  NOV  71)  PRR 

•  AD  SUPPLEMENT  TO  AFSCR  84-2  (2  JUN  77) 


PRODUCTION  READINESS  REVIEW 
OBJECTIVE 


•  READY  FOR  PRODUCTION 

•  PRODUCTION  ENGINEERING  PROBLEMS  RESOLVED 

•  ADEQUATE  PLANNING  ACCOMPLISHED 
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PRODUCTION  READINESS  REVIEW 
IMPORTANCE 

•  MAJOR  PROGRAM  MILESTONE 

•  RISK  ASSESSMENT  OF  CONTRACTOR’S  ABILITY 

•  PREREQUISITE  FOR  PRODUCTION  DECISION 


THE  PRR  PLAN 

•  A  FORMAL  DOCUMENT  GIVING 

•  PROGRAM  BACKGROUND 

•  TEAM  COMPOSITION  AND  RESPONSIBILITIES 

•  INSTRUCTIONS  TO  TEAM  MEMBERS 

•  AGENDA  OF  PRR 

•  POST  PRR  TASKS 

•  PRR  PLAN  APPROVAL  CYCLE 

•  AUTOMATED  DATA  BANK 
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WE  CAN  OUTFIT  A  TEAM  FOR  ANY  SITUATION 
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DATE 

TEAM 


INTERVIEWED  BY 


PERSON  INTERVIEWED 


(  )  SATISFACTORY _ (  )  UNSATISFACTORY 

QUESTION 


FINDINGS 


CONCLUSIONS/RECOMMENDATIONS 

(INCLUDE  RISK  ASSESSMENT  (HIGH.  MODERATE.  LOW) 
AND  WHY  AND  WHAT  MAY  BE  DONE  ABOUT  IT) 
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AREA  OF  CONCERN  [AOC) 
PROGRAM 
TEAM  MEMBER 


DATE 

PAGE  OF 


1. 

TOPIC 

2. 

THE  PROBLEM 

3. 

SUGGESTED  ACTION 

4. 

ACTION  TO  BE  TAKEN 

5. 

ACTION  DUE  DATE 

COMPLETED 
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OUR  TEAM  -  EVERYMAN  AN  ARMAMENTS  EXPERT 
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THE  PRR  TEAM 


TEAM  DIRECTOR 


DESIGN 

PANEL 


I - 

PROGRAM 

MANAGEMENT 

PANEL 


QUALITY  PROCUREMENT/ 

ASSURANCE  SUBCONTRACTING 
PANEL  PANEL 


TEAM  COORDINATOR 


MANUFACTURING 

PANEL 


NOTE:  PANELS  MAY  BE  AUGMENTED  BY  PERSONNEL  FROM  OTHER  AGENCIES 


USE  OF  MANUFACTURING  OPERATIONS  CONSULTANT 

•  MR  JACK  R.  FRANKS 
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CONDUCT  OF  THE  PRR 


•  PRIOR  NOTIFICATION  TO  GOVERNMENT  AGENCIES 

•  PRIOR  NOTIFICATION  TO  THE  CONTRACTOR 

•  INITIAL  BRIEFINGS 

•  DAILY  REVIEWS 

•  ROLE  OF  THE  PRINCIPAL  CONTRACTING  OFFICER 

•  FINAL  BRIEFINGS 

AD  HAS  RECEIVEO 
OUTSTANDING  SUPPORT 


FROM: 

AFCMD 

NAVY 

ARMY 


million 

FOUR. 

hundred 

THOU _ 


DEFENSE  LOGISTICS  AGENCY 
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REPORT  AND  BRIEFINGS 


•  FINAL  PRR  REPORT 

•  BRIEFING  TO  DEPUTY  FOR  CONTRACTING  ANO 
MANUFACTURING 

•  BRIEFING  TO  DEPUTY  FOR  ARMAMENT  SYSTEMS 

•  FOLLOW-UP  VISITS  AND  REPORTS 

•  BRIEFING  TO  AD  ACQUISITION  REVIEW  COUNCIL 


ARMAMENT  DIVISION  ORGANIZATION 
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INDEPENDENTLY 

BRIEFING  THE  PRR 

PREVENTS 

THE  SAME  OLD 

REACTION 

AND  DRAWS 

ATTENTION  TO  THE  RESULTS! 
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SAMPLE  PRR  QUESTIONS 


•  ARE  ALL  RELEVANT  FUNCTIONAL  DISCIPLINES  ADEQUATELY 
REPRESENTED  IN  THE  CONTRACTOR’S  REVIEWING  BODY 
FOR  PROPOSED  ENGINEERING  CHANGES? 

•  HAVE  OETAILEO  INSPECTION/TEST  METHOOS  ANO  PROCEDURES 
BEEN  PREPARED  AND  ACCEPTED  BY  THE  LOCAL  GOVERNMENT 
REPRESENTATIVE? 


SAMPLE  PRR  QUESTIONS  (CONT’D) 


•  HAS  DETAILED  PLANNING  BEEN  ACCOMPLISHED  TO  PRDVIDE 
FDR  AN  EFFECTIVE  MANPOWER  SUPPLY  OF  THE  PROPER 
NUMBER  AND  SKILLS  REQUIRED  FOR  PRODUCTION? 

•  ARE  THE  MASTER  PRODUCTION  SCHEDULES  COMPLETE  ANO  IN 
CONSONANCE  WITH  THE  MASTER  PROGRAM  SCHEDULE? 
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SLIDES  1/2  -  SHOCKED  FACE/WHAT  HAPPENS  WHEN  A  CONTRACTOR  FAILS? 

-  IN  THE  PAST  THREE  YEARS  THERE  HAVE  BEEN  TWO  CONTRACTORS  WHO 
FAILED  TO  PASS  PRRS  CONDUCTED  BY  AD  PERSONNEL. 

-  I  SERVED  AS  THE  TEAM  DIRECTOR  FOR  THE  PRR  I  AM  ABOUT  TO  BRIEF 
TO  YOU. 

-  WE  BELIEVE  THIS  PARTICULAR  PRR  IS  A  "CLASSIC"  FAILURE  ON  A 
contractor's  PART  BECAUSE  OF  THE  VARIETY  OF  DISCREPANT  AREAS  FOUND. 

slide  3  -  FMU-112/B 

-  THE  PROGRAM  IS  THE  FMU-112  IMPACT  SHORT  DELAY  FUZE. 

-  THE  PRR  WAS  A  CONTRACTUAL  REQUIREMENT  AND  WAS  CONDUCTED  AT  THE 
SCHEDULED  END  OF  FULL  SCALE  DEVELOPMENT. 

SLIDES  4  &  5  -  PICTURES  OF  THE  FUZE 

-  THESE  TWO  SLIDES  ARE  TO  GIVE  YOU  AN  IDEA  OF  THE  CHARACTERISTICS 
OF  THE  FUZE  WE  WILL  BE  DISCUSSING. 

-  THE  FMU-112  IS  A  POTTED  ELECTRONIC  FUZE.  IT  HAS  AN  AIR  DRIVEN 
POWER  SUPPLY,  PART  OF  WHICH- IS  ALSO  POTTED. 

-  THE  FUZE  MAY  BE  LOADED  INTO  EITHER  THE  NOSE  OR  TAIL  OF  A  BOMB 
AND  IS  COMPATIBLE  WITH  REMOTE  PROXIMITY  SENSORS. 

-  THE  FUZE  CAN  BE  USED  IN  A  HIGH  OR  LOW  DRAG  MODE.  TWO  ACCELEROMETERS 
ARE  BUILT  INTO  THE  FUZE  TO  SENSE  DRAG.  THESE  ACCELEROMETERS  ARE 
SMALL  CONTAINERS  WITH  A  STEEL  BALL  SITTING  ON  A  CALIBRATED  MAGNET. 
HIGH  DRAG  CONDITIONS  PULL  THE  BALL  OFF  THE  MAGNET  AND  AGAINST  A 

SET  OF  ELECTRICAL  CONTACTS,  A  CENTRIFUGE  IS  USED  IN  CALIBRATING 
THE  MAGNETS  TO  A  SPECIFIC  BREAKAWAY  FORCE  FOR  THE  STEEL  BALL. 

SLIDE  6  -  SUMMARY  OF  FINDINGS 

-  RETURNING  NOW  TO  THE  PRR,  THIS  NEXT  SLIDE  WILL  GIVE  THE  SUMMARY 
OF  FINDINGS  OF  THE  INSPECTION. 


247 


-  IT  SHOULD  BE  NOTED  THAT  THE  PRR  WAS  CONDUCTED  IAW  THE  PROCEDURES 
I  COVERED  IN  MY  PREVIOUS  BRIEFING, 

•  FIRST,  SOME  PERFORMANCE  REQUIREMENTS  WERE  NOT  MET  AS  OF  SEPTEMBER 
79,  BECAUSE  NOT  ALL  PROBLEM  CORRECTIONS  HAD  BEEN  TESTED  AND  THEIR 
SOLUTION  DEMONSTRATED  SATISFACTORILY,  THEIR  IMPACT  ON  PRODUCTION 
PLANNING  WAS  NOT  ADEQUATELY  KNOWN. 

•  SECOND,  MANUFACTURING  PLANNING  WAS  INCOMPLETE.  THE  APPARENT 
PREOCCUPATION  OF  PRGRAM  MANAGEMENT  WITH  TECHNICAL  PROBLEMS  DURING 
THE  DEVELOPMENT  CONTRACT  ALLOWED  INADEQUATE  PLANNING  TO  EXIST 
UNRESOLVED. 

•  THIRD,  MANUFACTURING  METHODS  AND  PROCESSES  REQUIRED  FURTHER 
DEVELOPMENT  TO  PREVENT  IMPACTING  LRIP  PROGRAM  COST  AND  SCHEDULE. 
THERE  WERE  TWO  MAJOR  PROBLEMS  IN  THIS  AREA. 

•  AT  THE  TIME  OF  THE  PRR,  THE  CONTRACTOR  DID  NOT  HAVE  A 
DEMONSTRATED  METHOD  FOR  POTTING  FUZES  OTHER  THAN  USING  SARAN 
MICROSPHERES,  WHICH  DOW  CHEMICALS  HAD  DISCONTINUED  MAKING, 

•  AT  THE  TIME  OF  THE  PRR,  PURCHASE  OF  TEN-STATION  CENTRIFUGES 
TO  CALIBRATE  DROGUE  ACCELEROMETERS  FOR  THE  PRODUCTION  EFFORT  WAS 
PLANNED,  BUT  THE  UNITS  HAD  NOT  BEEN  DEVELOPED  OR  DEMONSTRATED, 

•  FOURTH,  THE  CONTRACTOR'S  MATERIAL  MANAGEMENT  SYSTEM  DID  NOT 
IDENTIFY  AND  EMPHASIZE  CRITICAL  MATERIALS  OR  PURCHASED  ITEMS  OF 
FOREIGN  ORIGIN. 

•  FIFTH,  COSTING  METHODOLOGY  WAS  INADEQUATE  AS  THERE  WAS  NO 
SINGLE  FOCAL  POINT  OF  RESPONSIBILITY,  AND  THE  CONTRACTOR  HAD  NOT 
DEVELOPED  MANUFACTURING  OVERHEAD  AND  G  &  A  RATES  FOR  PRODUCTION. 

•  SIXTH,  THE  CONTRACTOR'S  PURCHASING  SYSTEM  DID  NOT  ADEQUATELY 
MANAGE  SUBCONTRACTORS  (DELINQUENT  AND  UNACCEPTABLE  DELIVERIES  NOT 
SYSTEMATICALLY  FOLLOWED~UP) .  THIS  WAS  A  SERIOUS  DEFICIENCY  AS 
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THIS  CONTRACTOR  IS  AN  ASSEMBLY  HOUSE  WITH  ALMOST  TOTAL  RELIANCE 
Oft  SUBCONTRACTORS  FOR  FABRICATION  AND  PARTS, 

SLIDE  7  -  CONCLUSION 

AS  A  RESULT  OF  THESE  FINDINGS  IT  WAS  CONCLUDED  THAT.,  AS  OF 
SEPTEMBER  1979,  THE  CONTRACTOR  WAS  NOT  READY  FOR  LRIP. 

SLIDE  8/9  -  CHRONOLOGY  OF  EVENTS/PERSONNEL'S  REACTION 

-  NOW,  let's  look  at  two  chronology  of  events  slides  and  see 

WHAT  HAPPENED  BETWEEN  THE  PRR  IN  SEPTEMBER  1979  AND  THE  AD/ACQUISITION 

REVIEW  COUNCIL  IN  AUGUST  1980. 

•  SEPT  79— CONTRACTOR  FOUND  NOT  READY  FOR  LRIP 

COMPANY'S  INITIAL  RESPONSE  TO  THE  PRR  REPORT  INADEQUATE, 
AND  WAS  REJECTED  BY  THE  GOVERNMENT. 

•  JAN  80— COMPANY'S  SECOND  RESPONSE  WAS  ACCEPTABLE,  WITH  THREE 
AREAS  OF  CONCERN  REMAINING: 

•  FIRST,  THE  CONTRACTOR'S  WRITTEN  RESPONSE  TO  THEIR  PURCHASING 
SYSTEM  INADEQUACIES  WAS  SATISFACTORY,  BUT  UNVERIFIED.  THE  PURCHAS¬ 
ING  SYSTEM  WILL  BE  REVIEWED  DURING  PAS  TO  INSURE  THAT  THE  STATED 

PROCEDURES  ARE  BEING  FOLLOWED. 

•  SECOND,  MANAGEMENT  OF  CRITICAL  MATERIALS  WILLBE  DONE  TO 
THE  GOVERNMENT'S  SATISFACTION  BY  INCLUDING  A  TAILORED  CLAUSE  FROM 
THE  AFSC  DAR  SUPPLEMENT  7~150.4  IN  THE  LRIP  CONTRACT. 

•  THIRD,  THE  REMAINING  MANUFACTURING  PROCESS  CONCERN  WAS 
DEVELOPMENT  OF  A  NEW  POTTING  COMPOUND  FORMULATION  AND  MECHANIZED 
POTTING  CAPABILITY. 

SLIDE  10/11  -  BATTERING  RAM/CHRONOLOGY  (cONT'd) 

•  FEB  80— GOVERNMENT  VISIT  TO  THE  CONTRACTOR 
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COMPANY  HAD  NOT  DONE  ANY  WORK  ON  NEW  FORMULATION 


AFWAL-ML  RECOMMENDED  THEY  DO  SO 

AFWAL-ML  WOULD  DO  AN  INDEPENDENT  FEASIBILITY  STUDY  OF 
contractor's  PROPOSED  APPROACH. 

AFWAL-ML  FIRST  ATTEMPTS  UNSUCCESSFUL 

•  MAY  80— AFWAL-ML  FINISHED  SECOND  ATTEMPT  AND  ISSUED  INITIAL 
FINDINGS:  DISAGREED  WITH  CONTRACTOR'S  SCHEDULE  AS  THEY  FELT 
DEVELOPMENT  OF  NEW  FORMULATION  NOT  A  STRAIGHTFORWARD  TASK. 

AFWAL-ML  FELT  IT  WOULD  TAKE  ALL  OF  A  YEAR. 

•  1  JUL  80— CONTRACTOR  STARTED  INDEPENDENT  DEVELOPMENT  OF  A 
PRELIMINARY  FORMULATION,  USING  THEIR  OWN  FUNDS. 

•  16  JUL  80— CONTRACTOR  COMES  TO  EGLIN  TO  DISCUSS: 

•  POTTING  COMPOUND  FORMULATION  DEVELOPMENT:  EFFORT 
EXPECTED  TO  BE  COMPLETED  BY  THE  THIRD  WEEK  IN  AUGUST 

•  DEVELOPMENT  OF  A  MECHANIZED  POTTING  CAPABILITY,  COMPANY 
PRESENTED  IN  DETAIL  HOW  THEY  WILL  DEVELOP  THIS  CAPABILITY. 
PRODUCTION  PERSONNEL  AGREED  WITH  THEIR  APPROACH. 

•  8  AUG  80— AD/ACQUISITION  REVIEW  COUNCIL  BRIEFED  ON  PROGRAM 
PRODUCTION  STATUS, 

SLIDE  12  -  CURRENT  STATUS 

-  THE  CURRENT  STATUS  (AS  OF  8  AUGUST  1980)  OF  THE  PRODUCTION 
ASPECTS  OF  THE  FMU-112  PROGRAM  WHICH  WERE  BRIEFED  TO  THE  AD/ARC 

IS  SHOWN  BY  THIS  SLIDE.  THE  BOTTOM  LINE  IS  THAT  FROM  A  PRODUCTION 
STANDPOINT,  THE  CONTRACTOR  WAS  READY  FOR  LRIP. 

-  IN  SUMMARY,  WE  CAN  LEARN  SEVERAL  VALUABLE  LESSONS  FROM  A  PRR 
SUCH  AS  THIS  ONE. 
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-  FIRST,,  THE  VALUE  OF  INCREMENTAL  PRRS.  THIS  PARTICULAR  PROGRAM 
DID  NOT  MAKE  USE  OF  INCREMENTAL  PRRS,  AS  WAS  STATED  EARLIER,  THE 
APPARENT  PREOCCUPATION  OF  PROGRAM  MANAGEMENT  WITH  TECHNICAL  PROBLEMS 
DURING  THE  DEVELOPMENT  CONTRACT  ALLOWED  INADEQUATE  PLANNING  TO  EXIST 
UNRESOLVED.  AN  INCREMENTAL  PRR  APPROACH  COULD  HAVE  PREVENTED  THIS, 

BY  POINTING  OUT  DEFICIENT  AREAS  EARLIER. 

-  SECOND,  THE  VALUE  OF  WORKING  THROUGH  AN  EVALUATION  SUCH  AS  THIS 
ONE  INDEPENDENTLY  OF  THE  PROGRAM  OFFICE.  THE  PRR  TEAM  HAD  MEMBERS 
FROM  THE  PROGRAM  OFFICE  AND  RESULTS  WERE  BRIEFED  TO  THE  PROGRAM 
OFFICE,  BUT  THE  PROGRAM  OFFICE  WAS  NOT  ALLOWED  TO  OVERRIDE  THE 
TEAM  DIRECTOR'S  FINAL  EVALUATION  OR  PREVENT  IT  FROM  BEING  BRIEFED 

TO  SENIOR  AD  MANAGEMENT.  THIS  INITIALLY  CAUSED  A  STRAINED  ATMOSPHERE, 
BUT  BY  THE  TIME  OF  THE  AD/ARC  THE  PROGRAM  OFFICE  REALIZED  THAT  THE 
INITIAL  FAILURE  OF  THE  CONTRACTOR  AND  WORK  DONE  AFTERWARD  ON  HIS 
PROBLEMS  ACTUALLY  ENHANCED  THE  PROGRAM  OFFICE  POSITION  BEFORE  THE 

ad/arc. 

-  AND  FINALLY,  THE  VALUE  OF  MAINTAINING  A  FIRM  BUT  HELPFUL  ATTITUDE 
TOWARD  THE  CONTRACTOR  PAYED  DIVIDENDS.  THE  CONTRACTOR  HAD  MIXED 
REACTIONS  TO  THE  OUTCOME  OF  THE  PRR.  HIS  FIRST  RESPONSE  TO  THE 
DEFICIENCIES  WAS  NOT  VERY  POSITIVE.  WHEN  IT  BECAME  OBVIOUS  TO  HIM 
THAT  THE  GOVERNMENT  CONSIDERED  THE  DEFICIENCES  SERIOUS  AND  WANTED 
CORRECTIVE  ACTION  BEFORE  GOING  ON,  AND  THAT  WE  WERE  WILLING  TO  WORK 
THE  PROBLEM  WITH  HIM,  HIS  ATTITUDE  CHANGED.  WE  WORKED  THE  PROBLEMS 
TOGETHER. 

SLIDE  13  -  COUNTING  FINGERS 

I  HAVE  NOW  COVERED  ALL  THE  POINTS  I  WISHED  TO  MAKE.  WE  HAVE  SEEN 
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HOW  AD  PREPARES  FOR  AND  CONDUCTS  PRRS.  WE  HAVE  ALSO  GONE  THROUGH 
A  SPECIFIC  PRR  WHICH  A  CONTRACTOR  FAILED.  THANK  YOU  VERY  MUCH  FOR 
YOUR  TIME  AND  ATTENTION.  ARE  THERE  ANY  QUESTIONS? 


252 


i 


WHAT  HAPPENS  WHEN  A 
CONTRACTOR  FAILS  A 
PRODUCTION  READINESS  REVIEW? 


MAJOR  STANLEY  ZALACE 
DIRECTORATE  OF  MANUFACTURING 
OEPUTATE  FOR  CONTRACTING  &  MANUFACTURING 
ARMAMENT  OIVISION 
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FMU-112/B 

IMPACT  SHORT  DELAY  FUZE 
PRODUCTION  READINESS  REVIEW 
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pRrsrNT  rMii-iw/R  ni7r  WFAPON  SYSTFM 


■ 


■ 


Nose  and/or  tail  compatible 
Use  for  high  drag  or  low  drag 
bomb  releases 
No  external  arming  wires 
Compatible  with  remote 
proximity  sensors 
Circuitry  compatible  with 
Mk  43  TDD 


PfliQ  JfIJHI T  V  StrtHQ  R 
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PRODUCTION  READINESS  REVIEW 
SUMMARY  OF  FINDINGS  AS  OF  SEPTEMBER  1979 


•  SOME  PERFORMANCE  REQUIREMENTS  NOT  MET 

•  MANUFACTURING  PLANNING  INCOMPLETE 

•  MANUFACTURING  METHODS  &  PROCESSES  REQUIRED  FURTHER  DEVELOPMENT 

•  MATERIAL  MANAGEMENT  SYSTEM  DID  NOT  IDENTIFY  CRITICAL  MATERIALS  OR 
PURCHASED  ITEMS  OF  FOREIGN  ORIGIN 

•  COSTING  METHODOLOGY  INADEQUATE 

•  PURCHASING  SYSTEM  DID  NOT  ADEQUATELY  MANAGE  SUBCONTRACTORS 

255 


PRODUCTION  READINESS  REVIEW 


CONCLUSION:  THAT  AS  OF  SEP  79.  THE  CONTRACTOR  WAS  NOT  READY 
TO  ENTER  LRIP  OF  THE  FMU-112/B  IMPACT  SHORT  OELAY  FUZE  AND 
THE  FZU-37A/B  BOMB  FUZE  INITIATOR 


CONTRACTOR  PERSONNEL  HAD  MIXED  REACTION  TO  THE  NEWS 
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CHRONOLOGY  OF  EVENTS 


24  26  SEP  79  PRR  CONDUCTED  AT  PLANT.  COMPANY  FOUND 

NOT  READY  FOR  LRIP 

17  JAN  80  PRR  FOLLOWUP  REPORT  ISSUED.  CONTRACTOR 

CONDITIONALLY  READY  FOR  LRIP 

PURCHASING  SYSTEM  INADEQUATE 

MATERIAL  MANAGEMENT  SYSTEM  INADEQUATE 

METHODS  &  PROCESSES  REQUIRE 
FURTHER  DEVELOPMENT 


WE  WERE  WELCOMED  BACK 
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CHRONOLOGY  OF  EVENTS  (CONT’D) 


25  FEB  80  MATERIALS  LABORATORY  &  PROGRAM  OFFICE  PERSONNEL 

VISIT  CONTRACTOR 

30  MAY  80  MATERIALS  LABORATORY  ISSUES  INITIAL  FINOINGS 

1  JUL  80  CONTRACTOR  STATEO  POTTING  COMPOUNO  DEVELOPMENT 

16JUL80  CONTRACTOR  MEETS  WITH  FMU-1 12  TEAM 

8  AUG  80  PRR  STATUS  BRIEFEO  TO  AO/ACQUISITION  REVIEW 

COUNCIL 


PRODUCTION  READINESS  REVIEW 
CURRENT  STATUS 


•  MANUFACTURING  PLANNING  SATISFACTORY 

•  MFG  METHODS  &  PROCESSES  PROPOSEO  ARE  LOW  RISK 

•  MATERIAL  MANAGEMENT  SYSTEM  APPEARS  SATISFACTORY  - 
TO  BE  VERIFIEO  OURING  PAS 

•  COSTING  METHODOLOGY  SATISFACTORY 

•  PROCUREMENT  SYSTEM  APPEARS  SATISFACTORY  - 
TO  BE  VERIFIED  OURING  PAS 

•  CONTRACTOR  IS  NOW  REAOY  FOR  LRIP 
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F-16  ASSESSMENT  OF  MULTI-NATIONAL  PRR's 
THE  IMPACT  OF  TECHNOLOGY  TRANSFER 


147L0 


LT  COL  THOMAS  D.  FIORINO 
DIRECTOR  OF  MANUFACTURING/QA 
DEPUTY  FOR  F-16 


150L0 


F-16  PROGRAM 


F-16  CO-PRODUCTION 

MAJOR  COMPONENT  RESPONSIBILITY 


GENERAL  DYNAMICS 
650  AIRCRAFT 


FOKKER 
174  AIRCRAFT 


SABCA 

174  AIRCRAFT 


261 


SEIECTED  SUBCONTRACTORS 


•  REVIEWED  LIST  OF  113  SUBCONTRACTORS 

•  SELECTED  53  FOR  GOVERNMENT  SURVEY 

•  SURVEY  RESULTS  INDICATED  REQUIREMENT  FOR  23  ON-SITE 
SURVEYS 

•  SELECTION  CRITERIA 

•  NEW  TECHNOLOGY 

•  CRITICALITY 

•  COMPLEXITY 

•  DOLLAR  VALUE 

•  COPRODUCTION 

•  TRACK  RECORD 
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KEY  POINTS 

OF  SUBCONTRACTOR  REVIEWS 


2017 


•  CONTRACTOR  DESCRIPTION 

•  HISTORY 

•  SIZE 

•  ORGANIZATION 

•  PRODUCT  MIX 

•  EXPERIENCE  FACTOR 

•  LABOR  RELATIONS 

•  GENERAL  CONTROL  SYSTEMS 

•  PROPERTY  CONTROL 

•  QUALITY  ASSURANCE 

•  PRODUCTION  CONTROL 

•  PURCHASING 

•  MAKE  OR  BUY 

•  FULL  SCALE  DEVELOPMENT  (FSD)  PROGRAM  STATUS 

•  SCHEDULES 

•  DELIVERIES 

•  TEST  SUMMARY/PROJECTED  COMPLETION 

•  DESIGN  STATUS/STABILITY 

•  PROBLEM  AREAS/CORRECTIVE  ACTIONS 
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KEY  POINTS  OF  SUBCONTRACTOR  REVIEWS  (CONTP) 

•  PRODUCTION  PROGRAM  PLANNING 

•  FACILITIES 

•  MANPOWER 

•  TOOLING/TEST  EQUIPMENT 

•  NON-STANDARD  PARTS 

•  CRITICAL/LONG  LEAD  ITEMS 

•  SOLE/SINGLE  SOURCE  ITEMS 

•  SIGNIFICANT  DIFFERENCES  FROM  FSD 

•  SCHEDULES 

•  GFAE 

•  REPAIR  CAPABILITY 

•  SUBSYSTEM  INTEGRATION 

•  PRODUCTION  RATE  CAPACITY 

•  HIGH  RISK  AREAS 

•  COPRODUCTION  PLANNING 

•  EUROPEAN  CONTRACTOR  DESCRIPTION 

•  COPRODUCED  HARDWARE 

•  DESCRIPTION  OF  COPRODUCTION  TASKS 

•  COPRODUCTION  SCHEDULES 

•  TRANSPORTATION/SHIPPING 

•  U.S. /EUROPEAN  COPRODUCTION  INTERFACE 

•  CONTINGENCY  PLANNING 
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F-16  PROGRAM 
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F-1 6  PROGRAM 


AREAS  OF  CONCERN 


«#» 


•  EUROPEAN  CO-PRODUCERS 

•  ADDED  LEAD  TIME 

SCHEDULING 
CHANGE  EFFECTIVITY 
TRANSPORTATION 

•  RESPONSE  CAPABILITY 

1  SHIFT  OPERATIONS 

•  LIMITED  ENGINE  MAINTENANCE 

•  FACTORY/AUTOMATIC  TEST  EQUIPMENT 

•  QUALITY  ASSURANCE 

•  DIFFERENT  MANAGEMENT  CONCEPTS 
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CONCLUSIONS 
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•  MULTINATIONAL  PRRs  ESSENTIAL 

•  FOLLOW-UP  IMPERATIVE 

•  STRENGTHENED  INTERFACES 

•  CONTRIBUTED  TO  OFFSET  MANAGEMENT 
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Presented  by: 

WILLIAM  R.  BRIGGS 
SD/PMDM 

SPACE  DIVISION  PRR  EXPERIENCES 

Much  of  what  I  have  to  say  has  been  covered  by  the  earlier  speakers;  however, 

I  would  like  to  address  Space  Division's  experience  with  Production 
Readiness  Reviews.  I  will  also  give  some  background  on  our  programs  and 
show  how  they  are  unique. 

CHART  #2 

Our  satellite  programs  typically  have  a  high  unit  cost.  They  are  complex  in 
that  we  are  continually  pushing  the  state-of-the-art  on  our  payloads.  We 
require  very  high  reliability  because  once  we  we  launch  them,  we  can't  get 
them  back.  Our  current  generation  satellites  have  a  10  year  design  life  and 
in  order  to  get  the  reliability  we  need,  we  have  a  space  qualified  electronics 
piece  parts  class  which  are  built  to  more  stringent  standards  than  Mil-Spec 
piece  parts.  Lastly,  our  programs  are  low  volume;  15  would  be  considered  a 
long  run.  Now  I  will  talk  about  the  Inertial  Upper  Stage  which  could  be 
considered  typical  of  our  programs,  then  I  will  discuss  the  planning  for  the 
Global  Positioning  System,  which  is  our  first  high  volume  program.  One  point 
I  would  like  to  make  is  that  we  don't  usually  go  into  the  classical  DSARC 
production  decision  process.  Since  we  fly  everything  we  buy,  we  are  essentially 
committed  during  full  scale  development;  therefore,  we  must  be  involved  early. 
CHART  #3 

This  is  a  breakdown  of  the  IUS  two-stage  vehicle.  It  has  kevlar  wrapped  motor 
cases  with  a  92"  diameter  first  stage  and  a  63"  second  stage.  It  is  an  aluminum 
skin  and  stringer  structure  with  redundant  avionics  to  avoid  single  point  failure 
modes . 
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The  I  US  is  also  used  as  an  upper  stage  on  the  Titan  III  launch  vehicle.  The 
IUS  can  be  converted  from  shuttle  to  Titan  use  if  the  need  is  identified  early 
in  the  production  flow. 

CHART  #5 

This  is  our  first  PRR  and  three  years  ago  this  was  the  wording  that  was  placed 
in  the  Statement  of  Work.  Notice  that  there  is  no  contractual  flow  down  to 

subcontractors.  However,  the  prime  contractor  and  subcontractors  were  cooperative 
in  conducting  the  PRRs. 

CHART  #6 

This  viewgraph  illustrates  the  changes  that  have  occurred  in  the  program  since 
we  originally  envisioned  doing  the  PRR.  You  all  have  heard  that  the  shuttle 
schedule  has  slipped,  payload  schedules  have  also  slipped.  Originally  the 
shuttle  was  to  launch  ahead  of  Titan.  The  requirement  for  the  Titan  is  because 
some  payloads  were  not  compatible  with  shuttle  launch  loads  and  therefore  have 
to  be  launched  on  the  Titan. 

CHART  #7 

After  a  full  scale  development  contract  was  awarded,  we  had  determined  that 
22  critical  design  reviews  were  scheduled  from  Dec  78  to  Jul  79.  We  naturally 
felt  that  all  contracted  components  did  require  a  PRR  so  we  scoped  the  reviews 
to  cover  the  larger  of  our  critical  subcontractors .  Eleven  components  were 
viewed  as  critical  to  the  program  and  we  conducted  eight  incremental  reviews 
to  cover  them.  The  prime  contractor  had  the  responsibility  to  conduct  the  PRR 
at  the  subcontractors,  the  Air  Force  participated  as  advisory  team  members. 

CHART  #8 

When  we  did  the  Systems  PRR  at  Boeing,  we  baselined  the  design  at  the  DOD 
Two-Stage  with  the  scheule  presented  here.  One  point  I  like  to  illustrate  is 
that  the  FSD  production  rate  is  higher  than  the  follow-on  production,  which  points 
out  the  uniqueness  of  our  programs. 


CHART  #9 


Here  is  a  surmary  of  the  reviews  we  conducted  and  some  of  the  discrepancies 
noted.  The  details  aren't  important,  however  some  action  items  were  written 
against  the  prime  or  first  subcontractor.  Our  early  involvement  pointed  out 
in  several  instances  that  material  had  not  been  ordered  in  time  to  support 
the  manufacturing  schedule.  Another  concern  was  the  high  degree  of  schedule 
concurrency  with  the  qualification  model  parts  and  flight  vehicle  parts. 

This  approach  has  some  inherent  risk  but  was  required  to  maintain  schedules. 
CHART  #10 

Now  I  would  like  to  talk  about  the  Global  Positioning  System,  which  is  one  of 
our  largest  programs.  The  program  consists  of  a  space  segment  of  18  satellites 
in  3  orbital  planes,  a  user  segment  which  has  equipment  to  be  installed  on 
aircraft,  ships,  vehicles,  and  a  manpack;  and  the  ground  control  system  segment 
which  consists  of  monitored  stations,  a  master  control  station  for  satellite 
command  and  control,  and  an  upload  data  link. 

CHART  #11 

Here  is  a  list  of  applications,  but  the  basic  mission  is  precise  navigation 
on  a  worldwide  scale. 

CHART  #12 

This  is  a  photograph  of  the  different  configurations  to  be  found  in  the  user 
equipment  family.  The  total  volume  is  somewhere  between  20  and  30  thousand 
units,  depending  upon  the  equipment  mix. 

CHART  #13 

The  program  schedule  depicted  here  will  show  you  the  relationship  the  three 
segments  have  to  one  another. 
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CHART  #14 


The  factors  that  were  considered  in  the  PRR  planning  for  the  user  equipment 
are:  the  size  of  the  program  which  is  the  largest  at  Space  Division;  a  three 
year  full  scale  development  contract  which  is  a  tri -service  program  with 
possible  NATO  participation;  and  two  contractors  in  competition  throughout 
the  FSD  phase.  We  have  good  visibility  into  the  contractors  operations  and 
we  did  a  Manufacturing  Management/Production  Capability  Review  prior  to  the 
awarding  of  the  FSD  contract. 

CHART  #15 

The  PRR  Plan  is  the  classical  one  in  that  we  will  do  incremental  reviews  at 
the  prime  contractors  as  well  as  critical  subcontractors.  Through  our  early 
involvement  we  are  emphasizing  design  and  testing  readiness  as  well  as  the 
producibil ity  of  the  equipment.  The  review  will  be  conducted  using  small 
teams  to  maintain  personnel  continuity  relying  heavily  upon  the  joint  program 
office  resources. 

CHART  #16 

We  address  much  of  the  design  criteria  in  conjunction  with  the  scheduled 
design  reviews  as  well  as  the  parts,  materials  and  processes  control  board. 

We  also  have  a  logistics  interface  which  will  be  evaluated  by  the  Logistics 
Directorate  in  the  program  office. 

CHART  #17 

The  PRR  as  written  uses  these  criteria  and  other  areas  of  concern. 

CHART  #18 

The  incremental  reviews  are  structured  to  evaluate  these  areas  which  have  a 
logical  phasing  to  the  program  schedule.  Software  is  also  addressed  in  the 
design  reviews.  Criteria  "G"  addresses  standardization. 


Now,  let's  discuss  the  Space  Segment  and  our  initial  planning  for  the  PRR. 

CHART  #20 

The  spacecraft  contractor  has  had  some  experience  in  that  during  the 
validation  phase,  which  we  are  presently  in,  the  contractor  has  built 
and  launched  six  satellites  for  our  test  program.  Schedule  was  also  a 
determining  factor  in  our  planning  since  we  anticipate  a  production  contract 
award  early  in  1982.  The  program  also  has  some  funding  restrictions  which 
caused  us  some  problems  in  scoping  the  incremental  reviews. 

CHART  #21 

We  plan  to  do  incremental  reviews  at  the  prime  contractors  as  well  as  10  critical 
subcontractors.  The  prime  contractor  has  the  responsibility  to  review  the 
remainder  of  the  subcontractors  without  the  AF  participation  in  the  reviews. 

We  will  assess  the  criteria  listed  in  DOD  Instruction  5000.38. 

CHART  #22 

We  have  found  that  early  involvement  in  the  PRR  process  is  essential  for 
program  visibility  and  to  provide  management  an  early  assessment  of  the 
program  status.  The  evaluation  criteria  listed  in  AFSCR  84-2  has  to  be 
tailored  to  fit  the  unique  environment  of  space  systems.  The  contractual 
tasking  of  the  prime  should  be  as  specific  as  possible  and  should  also  entail 
flow  down  to  critical  subcontractors.  We  also  found  that  it  is  necessary  to 
provide  explicit  tasking  for  the  Contract  Administration  Office  involvement 
and  may  entail  pre-review  meetings  to  insure  a  good  understanding  by  the  CA0 
and  contractor  as  well.  One  point  made  earlier  is  that  a  coordinated  agenda 
is  essential  for  the  conduct  of  the  review.  If  you  are  reviewing  a  large 
contractor  who  has  had  PRRs  in  the  past,  it  would  be  helpful  to  know  how  the 
contractors  fared  in  those  reviews.  This  approach  would  enable  us  to  baseline 
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the  contractor  management  system  as  well  as  identify  areas  of  concern  relevant 
to  the  program  under  review.  The  Contract  Administration  Office  would  be  an 
excellent  source  for  the  management  system  baseline.  Then  the  review  team 
could  evaluate  the  program  peculiar  requirements  in-depth.  Lastly,  team 
continuity  is  essential  in  efficiently  conducting  the  review.  You  don't  have 
to  get  new  team  members  up  to  speed  on  the  PRR  process. 

Thank  you  for  your  attention. 
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CONTRACT  REQUIREMENT; 


"PRODUCTION  READINESS  REVIEWS  (PRR) .  A  PRR  SHALL  BE 
ACCOMPLISHED  60  DAYS  AFTER  THE  CRITICAL  DESIGN  REVIEW." 
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WHAT  IS  GPS? 


USER  SET  CLASSES 
FOR  ALL  DOD  USERS 

•  AIRCRAFT 

•  SHIPS 

•  TRACK/WHEELED  VEHICLES 

•  MANPACK 

•  UNMANNED  APPLICATICNS 
STATIC  AND  DYNAMIC  USERS 
HI-LOW  PERFORMANCE/COST  Ml 
COMMON  MODULES 


*  24  SATELLITES 

*  10.900  N  ML  ORBITS 

•  12  HR  PERIOD 

*  3  ORBITAL  PLANES 


*  2  L  BAND  FREQUENCIES 

•  1227  MHz 

•  1575  MHz 


SATELLITE  GROUND  CONTROL  SYSTEM 

•  MONITOR  STATIONS 

•  MASTER  CONTROL  STATION 

•  UPLOAD  STATION 

SATELLITE  COMMAND  AND  CONTROL 
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GPS  APPLICATIONS 


ENROUTE  NAVIGATION 
LOW  LEVEL- NAVIGATION 
TARGET  ACQUISITION 
CLOSE  AIR  SUPPORT 
MISSILE  GUIDANCE 
COMMAND  AND  CONTROL 
ALL  WEATHER  AIR  DROP 
SENSOR  EMPLACEMENT 
PRECISION  SURVEY 
SPACE  SHUTTLE  NAVIGATION 
IONDS 


INSTRUMENT  APPROACH 

RENDEZVOUS 

COORDINATE  BOMBING 

RPV  OPERATIONS 

BAREBASE  OPERATIONS 

SEARCH  AND  RESCUE 

PHOTO  RECONNAISANCE 

PASSIVE  ELINT 

SATELLITE  TRACKING 
AND  NAVIGATION 


INS  UPDATE 
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SD  FONM  200  PREVIOUS  [DITION  IS  OBSOLETE.  SD  JAN  90  200  PREVIOUS  EDITION  IS  OBSOLETE, 
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ENGINEERING/PRODUCT  DESIGN  SOFTWARE 

•  PRR  n 

MANUFACTURING  MANAGEMENT 
MATERIAL  MANAGEMENT 

•  PRR  # 2 

FACILITIES 

TOOLING 

TESTING 

•  PRR  #3 

EQUIPMENT 
ORGANIZATION 
t  PMPC  BOARD 

CRITERION  "G" 

•  LOGISTICS  DIRECTORATE 

LOGISTICS  CRITERION 
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$D  r0RM  200  PREVIOUS  EOmOM  IS  ORSOLE  TC.  $D  FORM  200 
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SPACE  SEGMENT 
MAJOR  SHAPING  FACTORS 

•  CONTRACTOR  EXPERIENCE 

•  SCHEDULE 

•  FUNDING 
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LESSONS  LEARNED 


EARLY  PRR  INVOLVEMENT  AMD  FOLLOW  UP 
TAILOR  AFSCR  84-2  CRITERIA  FOR  SPACE  SYSTEMS 
CONTRACTUAL  FLOWDOWN  TO  CRITICAL  SUBCONTRACTORS 
EXPLICIT  TASKING  FOR  CAO  INVOLVEMENT 
FEEDBACK  FROM  OTHER  PRR's 
TEAM  CONTINUITY 


afcmd/afpro  presentation 

DOD  CONFERENCE 
PRODUCTION  READINESS  REVIEW 


The  AFPRO  at  the  Hughes  Aircraft  Company  has  had  extensive 
experience  in  Production  Readiness  Reviews  (PRR). 

Chart  1 


During  the  last  fiscal  year,  we  have  participated  in  18  PRR's. 
Seven  (7)  involved  various  Army  programs,  3  were  for  the  Navy  and  3 
for  the  Air  Force.  Two  of  the  reviews  were  fragmented  into  2  or  more 
incremental  visits.  The  total  manpower  expended  by  the  AFPRO  was 
3,040  man/hours  averaging  about  2  engineers  dedicated  to  the  PRR  team 
for  two  weeks.  The  cost  of  the  AFPRO  participation  was  $68,000  -  none 
of  which  was  charged  to  the  buying/program  office. 

The  manner  and  extent  of  the  AFPRO  participation  and  the  observa¬ 
tions  made  as  a  result  of  this  participation  will  be  discussed  later. 

The  next  chart  (Chart  2)  identifies  the  various  programs  that  were 
subjected  to  PRR's  at  Hughes.  The  F-18  and  the  U.S.  Roland  programs 
were  accomplished  by  incremental  review,  of  2  or  more  visits.  All 
others  were  undertaken  as  single  visit  PRR's  from  one  to  four  weeks 
duration. 
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Chart  2 
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PROGRAMS  BUBJICTID  TO  PRRS 

AIR  FORC* 

•  P-1S  AAOAR  IMtTSTCN  (MAC* IR /ASO>  •  JOINT  TACTICAL  INFORMATION  OISTAIRUTIOM  SYSTEMfE  SO) 

•  MU-11  TAR6CT  DETECTORS  (VRALC) 

ARMY 

•  (ROUND  LASCR  DESIGNATOR  SYSTEM  (MIRADCOM)  •  U.S.  ROLANO  FIRC  CONTROL  t  MISSILES  (MIRADCOM) 

•  JtM-|  TANK  P  IRC  CONTROL  STSTCN  (TACOM)  •  FI6HTIMC  VEHICLE  SYSTEM  (TARAOCOM)  •  LASER  TAR6CT 
OCTCCTO*  (MIRADCOM)  •  LICMTWtICNT  LAUNCHER  (MIRCOM)  •  FIRCFIMOCR  (CRADCOM) 

NAVY 

•  AS  TARGET  RECOGNITION  ATTACH  FM.TISCMSOR  (MAVAIR  JTJCOM)  •  DSU-tS  TARGE  T  M TEC TOR  (NAVAIR) 

•  P-tl  RADAR  SUBSYSTEM  (MAC |  Mi -MAVAIR ) 


The  manner  and  extent  of  the  AFPRO  participation  in  PRR's  is 
promulgated  in  the  AFPRO  regulation  84-3,  which  implements  the  re¬ 
quirements  of  the  Air  Force  AFSCR  84-2,  and  Chapter  6  of  AFCMDR  84-1. 

We  have  and  will  assist  the  buying/program  office  in  the  pre¬ 
liminary  PPR  actions,  in  on-site  reviews  and  post  PRR  activities. 

The  chart  identifies  these  functions. 

Chart  3 
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How  these  functions  interrelate  with  the  Program/Buying  office 
is  shown  on  this  chart. 


Chart  4 
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OBSERVATIONS 


AfPtO  KNOWtCMC  Of  TIC  CONTRACTOR  t  EXPERTIZE  NOT  USED  EFFECT  I VELT 
SIHCIE  VISIT  TEAMS  TOO  LAI(E  -  UPSET  CONTRACTOR'S  OPERATIONS 
tNQUAL IP ItO  PERSONS  INCLUDED  ON  PRR  TEAMS 
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CONTRACTOR'S  CHARLES  FOR  PRR  EFFORT  SHOULD  NOT  RE  ACCEPTED 
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Some  of  the  Buying/Program  offices  were  unaware  of  an  AFPRO's 
knowledge  of  the  contractor's  facilities,  capabilities  and  operations. 
We  were  frequently  asked  to  furnish  evaluation  manpower  without  much 
advance  notice  and  only  once  were  we  asked  to  participate  in  pre-PRR 
action.  On  the  F-18  review,  the  Navy  requested  us  to  review  and 
comment  on  their  PRR  plan. 

At  this  AFPRO  we  have  developed  and  maintain  a  Master  PRR  File. 

It  is  current  and  reflects  the  contractor's  production  readiness  in 
fifteen  (15)  functional  areas  and  related  factors. 


A  contractor's  condition  of  production  readiness  in  each  factor 
is  determined  by  his  compliance  to  criteria  questions. 

About  90%  of  the  criteria  questions  are  systems  oriented  and  apply 
to  all  programs.  Under  our  Master  File  concepts,  this  criteria  has 
been  previously  evaluated  and  a  PRR  team  need  only  to  concentrate  on 
those  criteria  (10%)  which  apply  to  a  specific  program.  In  the  Make-or- 
Buy  functional  area,  there  are  nineteen  criteria  questions,  four  of 
which  apply  to  a  specific  program.  For  example,  the  question:  "Does 
the  contractor  have  a  Make-or-Buy  Program  which  is  in  compliance  with 
the  requirements  of  DAR  3-9000?"  is  applicable  to  all  programs  and  the 
response  is  the  same  for  every  program.  The  question:  "What  percentage 
of  the  program  is  make  and  what  percentage  is  buy?  applies  to  a  specific 
program  and  should  be  considered  by  a  Program  PRR  Team. 
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Contact  with  the  AFPRO  prior  to  an  on-site  review  or  excess  to 
our  Master  File  can  reduce  the  evaluation  effort  considerable. 


Large  single  visit  teams  containing  as  many  as  60  persons  have 
been  upsetting  to  the  contractor's  operations.  Last  November  the 
Army  and  Navy  conducted  two  PRR's  simultaneously  with  over  150 
"visiting  firemen"  in  the  same  plant.  Incremental  PRR's  are  more 
productive  and  less  disturbing  to  the  contractor. 


Chart  6 
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Such  a  review  could  be  fragmented  into  4  congenerous  functional 
areas  and  teams  of  4  to  5  persons  could  evaluate  these  areas  at  different 
intervals  of  time.  Another  advantage  that  becomes  apparent  is  that  a 
review  of  the  corrective  actions  required  by  a  previous  review  could  be 
made  in  a  subsequent  review. 
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APCMD  37  GENERAL  PURPOSE  PRESENTATION  CHART 


PRR  CONFERENCE  CLOSING  VIEWS  AND  CONTINUING  ACTIONS 


Truxtun  R.  Baldwin,  Conference  Chairman 


The  panel  has  nominated  me  to  be  their  spokesman.  So  I'm  going  to  speak  for  all,  but  I'm 
sure  they'll  jump  in  if  there's  something  that  I  didn't  cover.  I'd  like  to  go  back  to  some  of 
the  questions  and  issues  that  I  posed  when  we  opened  the  conference  yesterday.  One  of 
the  questions  was,  "How  do  Services  bring  production  management  emphasis  to  bear 
earlier  in  the  acquisition  cycle?"  I'm  convinced  after  what  we've  gone  through  here  in  the 
last  two  days  that  we  are  doing  this  and  that  the  PRR  is  a  very  useful  vehicle  and  an 
authoritative  vehicle  for  providing  us  the  impetus  to  do  so. 

We  had  another  question.  "Are  we  spending  the  right  amount  on  PRRs?"  We  gave  you  the 
range,  both  in  absolute  dollars  and  in  percentage  of  development  cost  from  Gary  Dillard's 
paper.  I  haven't  heard  anyone  suggest  we  are  spending  too  much.  We  don't  have  any 
contractors  here  at  this  particular  session  to  voice  their  opinion,  but  the  conference  as  a 
whole  has  provided  us  with  some  insight  on  how  we  can  make  some  economies  in  cutting 
down  the  number  of  visits,  the  size  of  the  teams,  better  use  of  the  contract  administration 
office,  its  residents.  We  have  many  avenues  pointed  out  to  us  where  we  can  economize. 

"Are  we  making  PRR  preparations  and  procedures  easier  for  the  contractor?"  I  think  the 
avenue  to  that  is  more  thorough  planning  and  that's  been  a  key  recommendation  of  just 
about  every  speaker  that  we've  had.  We  have  recommendations  to  not  require  any 
additional  data  of  the  contractors,  and  we  have  such  contract  clauses  included  already  in  a 
number  of  cases,  and  again  we're  limiting  the  size  of  teams  and  the  number  of  team 
members. 

Another  question  was,  "How  can  we  improve  the  credibility  of  PRRs  by  quantifying  our 
findings?  "  One  of  the  continuing  actions  which  I'll  be  going  over  shortly  with  you  was  that 
there's  a  consensus  that  DPESO  should  work  more  on  these  PRR  indicators.  They'll  be 
needing  more  data  to  do  this  and  we'll  be  requesting  that  of  you,  as  you'll  see  shortly. 

The  final  question  was,  "What  new  definitions  or  guidelines  or  other  changes  are  necessary 
to  DoD  Instruction  5000.38  and  how  should  they  be  transmitted  to  the  DoD  components? 
We've  discussed  some  new  guidance,  particularly  in  the  area  of  software.  I  think 
something  is  in  order  there,  and  I  would  like  to  see  the  conference  proceedings  stand  as 
the  means  of  disseminating  this.  I  think  we  would  be  ill-advised  to  rush  in  and  try  to 
change  a  document  such  as  the  Instruction  every  time  we  have  a  single  change  or  addition 
that  we  need  to  make.  Since  the  Instruction  is  general  guidance  in  nature,  we  can  let  the 
proceedings  handle  that  for  us. 

Overall,  as  I  look  at  the  conference,  it's  been  very  valuable  and  that's  been  by  and  large 
the  main  comment  that  I've  gotten  from  everyone  that  I  have  talked  to.  It's  very 
complimentary  to  the  production  community  as  a  whole  that  we  can  hold  a  very  orderly 
and  a  very  good  exchange  of  information  and  share  our  experiences  with  each  other. 

I'd  like  to  give  particular  plaudits  to  the  people  that  have  administered  this  session;  the 
Air  Force,  as  you  know,  has  been  our  very  gracious  host.  We  put  them  last  on  the  agenda 
not  because  of  any  alphabetical  or  seniority  order,  but  we  thought  you  would  insist  as  a 
gracious  host  that  you  do  come  last.  We  are  very  much  indebted  to  you  for  holding  this 
and  Lt.  Goodale  has  seen  for  our  every  need.  He's  been  able  to  anticipate  everything  that 
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would  be  needed.  It's  one  of  the  most  efficient  jobs  that  I  have  seen  of  putting  on  a 
conference,  from  the  PA  system  to  every  furnishing  and  almost  every  creature  comfort 
that  we  could  hope  for.  On  our  DPESO  side  Lee  Schumacher,  as  you  know,  has  honchoed 
that  aspect  of  the  thing,  and  Mary  Hart  is  back  there  as  his  right-hand  person.  I 
particularly  want  to  thank  my  panel  members  who  have  given  me  such  able  assistance. 
Everyone  who  has  administered  this  conference  is  owed  a  debt  of  gratitude.  It's  been  very 
efficiently  done  and  we've  gotten  maximum  mileage  out  of  our  time  here  because  of  it. 

I'd  like  to  depart  from  the  agenda  now.  Instead  of  going  into  an  open  discussion,  I'd  like  to 
go  through  some  of  the  continuing  action  items  that  I  have  identified  and  worked  up 
overnight  to  put  before  you.  I  have  about  a  dozen  or  so  of  them  and  what  I  would  like  to 
do  is  go  through  this  list.  I'll  present  an  observation  as  to  what  I  saw  as  a  key  item  from 
my  minutes,  and  then  I'll  propose  a  continuing  action.  In  a  couple  of  cases  you'll  see  I  have 
no  action  put  down,  because  although  I  wanted  to  recognize  the  discussion  that  took  place, 

I  thought  the  consensus  was  that  we  should  not  follow  through  and  take  some  action.  So 
let  me  start  up  at  the  top.  These  are  only  in  the  order  that  my  minutes  reflect. 

Observation  //I 

Defense  Systems  Management  College's  (DSMC)  Program  Management  Course  does 
not  contain  material  on  planning,  organizing,  and  conducting  PRRs. 

Action  //I 

OSD  (AP)  MSA  will  provide  course  and  lecture  material  for  DSMC  use.  Note:  DSMC 
has  contracted  for  the  preparation  of  a  Program  Manager's  Guide  for  Production 
Management  which  will  have  a  section  on  PRRs  (see  appendix  B). 

Observation  #2 

Credibility  of  PRR  findings  and  support  for  PRR  conclusions  and  recommendations 
would  be  improved  by  more  quantitative  data. 

Action  #2 

DPESO  will  continue  its  work  in  development  of  baseline  indicators  from  normalized 
data  having  statistical  significance. 

Observation  //? 

DPESO  work  on  PRR  baseline  indicators  is  preliminary  in  nature  due  to  small 
amount  of  data  available. 

Action  //3 

DPESO  will  prepare  a  request  for  sharing  existing  data  to  be  included  in  PRR 
Conference  Proceedings  for  response  by  conferees.  Service  PESOs  will  assist  in  collection 
of  data.  (Note:  See  page  295  of  these  proceedings.) 

Observation  #4 

Integral  or  embedded  software  associated  with  operation  and  support  of  the  system 
is  growing  in  magnitude,  complexity,  cost  and  impact  on  program  readiness  for  production 
but  is  not  given  specific  treatment  in  DoD  Instruction  5000.38,  Appendix  A  "Production 
Readiness  Guidelines." 

Action  //4 

DPESO  will  prepare  appropriate  wording  for  a  new  item  A  9  under  Product  Design 
for  dissemination  with  the  conference  proceedings.  (Note:  See  page  297  of  these  proceedings.) 
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Observation  #5 

More  advance  notice  and  schedule  coordination  is  required  between  DPESO  and  the 
sponsoring  Service  on  DPESO  attendance  and  participation  in  Service-conducted  PRRs. 

Action  #5 

DPESO  will  meet  with  the  Service  PESOs  to  advise  which  of  their  programs  are 
identified  as  major  systems  by  OSD,  and  to  set  preliminary  PRR  schedule  and  tentative 
DPESO  participation. 

Observation  #6 

There  is  no  DoD  PRR  plan  equivalent  to  the  Army  AVRADCOM  document. 

Continuing  Action  #6 

None  -  such  a  document  would  be  cumbersome  considering  the  wide  differences  in 
products,  acquisition  concepts  and  approaches  to  PRRs  held  by  the  services.  Individual 
commands  and  activities  within  the  services  are  encouraged  to  formalize  their  PRR 
planning  process  as  they  deem  beneficial. 

Observation  #7 

The  Services  vary  in  their  selection  of  a  PRR  team  leader  ("director  and  focal 
point")  between  a  member  of  the  PM  office  and  an  independent  activity.  DoD  Instruction 
5000.38,  paragraph  F  1  does  not  so  distinguish. 

Action  #7 

None  -  such  selection  responsibility  is  delegated  to  the  DoD  component.  Objectivity 
is  served  as  a  consequence  of  the  wide  ranging  technical  expertise,  Service  PESO  and 
DPESO  participation  and  the  independent  production  readiness  assessment  prepared  for 
the  DSARC  Chairman. 

Observation  //8 

Experiences  gained  in  conducting  PRRs  to  date  would  be  beneficial  to  others  holding 
PRRs  in  the  future. 

Action  #8 

DPESO  in  conjunction  with  Service  PESOs  will  glean  major  "lessons  learned"  from 
presentation  material  and  discussions,  list  by  "PRR  Conference  Subject  Areas"  and 
distribute  with  conference  proceedings.  (Note:  See  page  297  of  these  proceedings.) 

Observation  #9 

Government  plant  activities  can  be  of  still  greater  assistance  to  PRR  teams  in 
conducting  PRRs  if  ample  notification  and  assistance  desired  are  communicated. 

Action  #9 

Service  PESOs  in  planning  and  scheduling  PRRs  for  their  Service  will  assure  that  the 
cognizant  plant  activities  for  prime,  sub,  and  GFE  contractors  are  advised  of  the  schedule 
and  scope  of  PRRs. 

Observation  #10 

The  DoD  PRR  Conference  was  most  worthwhile  and  should  be  held  annually. 

Action  #10 

Navy  PESO  will  take  into  consideration  hosting  a  meeting  next  year  which  will 
follow-up  on  this  year's  conference  and  possibly  embrace  a  second  topic,  e.g.,  "Production 
Engineering  and  Planning."  Contractor  participation  would  also  be  considered. 
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Observation  #11 

~OSD/DPESO  Production  Readiness  Assessments  (PRAs)  are  reviewed  with  the 
applicable  PM  offices  but  not  necessarily  with  other  participating  elements. 

Action  #11 

Subsequent  to  issuance  of  the  Secretary  of  Defense  Decision  Memoranda,  DPESO 
will  provide  a  signed  copy  of  the  PRA  to  the  PM  and  to  the  Service  PESO  of  the  cognizant 
Service.  b 

Observation  #12 

Cost  investigations  could  be  better  used  to  identify  potential  production  problems 
and  resulting  schedule  and  cost  risk. 

Action  #12 

DPESO  and  Service  PESO  participation  in  PRRs  will  exercise  Design-to-Unit- 
Production-Cost  and  5000.38  criterion  A-7  to  advantage  during  PRRs  to  identify 
troublesome  areas  and  components. 

Observation  #13 

Cost  reduction  is  a  major  concern  of  the  PRR.  Plans  and  provisions  for  cost 
reduction  during  production  are  to  be  presented  to  the  DSARC  at  Milestone  III  (per  DoD 
Directive  5000.34). 

Action  #13 

DPESO  will  critique  the  system's  cost  reduction  program  in  preparing  the  Production 
Readiness  Assessment. 

Observation  #14 

The  Production  Management  community  needs  a  PRR  data  bank  to  include  lessons 
learned  and  contractor  assessments. 

Action  #14 

Army  PESO  in  conjunction  with  DPESO  and  other  Service  PESO'S  will  take  this  item 
under  study  to  include  the  possibility  of  establishing  Army  Management  Engineering 
Training  Activity  training  program.  ° 
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SHARING  OF  PRODUCTION  READINESS  REVIEW  DATA 


The  DoD  Product  Engineering  Services  Office  (DPESO)  is  conducting  studies  to 
quantify  various  measures  of  production  readiness.  (Refer  to  "Baseline  Indicators  of 
Production  Readiness"  and  "Production  Readiness  of  Computer  Software,"  pages  25  and 
31  in  these  proceedings.)  The  output  of  the  studies  will  include  statistical  distributions 
depicting  how  values  of  selected  readiness  indicators  vary  as  a  function  of  program 
maturity.  Discussions  during  the  Production  Readiness  Review  (PRR)  Conference 
indicated  that  these  distributions  would  be  of  value  in  making  objective  determinations  of 
production  readiness. 

Further  progress  in  developing  these  distributions  would  be  greatly  expedited  if  more 
quantitative  data  were  made  available  during  the  study  effort.  Accordingly,  contributions 
of  appropriate  data  are  solicited  from  all  individuals  involved  in  PRRs.  Submission  of  this 
data  is  exempt  from  the  review  and  approval  provisions  of  DoD  Directive  5000.19, 
Enclosure  3  because  it  constitutes  voluntary  sharing  of  information  obtained  and  evaluated 
in  the  normal  course  of  the  PRR  process. 

Although  contributions  of  data  relating  to  any  quantifiable  indicators  of  production 
readiness  will  be  welcomed,  there  is  immediate  need  for  data  in  the  following  categories: 

•  For  Hardware: 

•  Engineering  change  activity  (with  an  indication  of  the  number  of 
drawings  which  comprise  the  total  drawing  package) 

•  Reliability  growth 

•  Yield  rates  for  manufacturing  processes 

•  Yield  rates  for  test  operations 

•  Scrap  and  rework  rates 

•  Out-of-station  work 


•  For  Software: 


•  Extent  of  total  computer  memory  utilized 

•  Extent  of  total  computer  processing  time  utilized 

•  Software  error  discovery  rate 

•  Software-related  change  activity  (if  possible,  separate  changes  to 
specifications  from  changes  needed  to  correct  baselined  software) 

•  Mean  time  between  computer  program  errors  (during  formal  testing) 


There  is  no  need  for  the  data  to  be  provided  through  formal  channels,  nor  in  a 
particular  format.  Long  term  trend  data  is  preferred  in  lieu  of  point  values.  Please 
identify  the  data  by  including  the  following: 


#  Name  or  nomenclature  of  the  product  and/or  process 

•  Corresponding  significant  events  or  phases  in  the  life  cycle  of  the  program  or 
product  (i.e.,  prototype  build  period,  development  engineering  testing,  opera¬ 
tional  testing,  software-hardware  integration,  production  release,  etc.) 


Please  include  the  name,  organization,  and  telephone  number  of  the  individual  submitting 
the  data  to  facilitate  any  needed  clarification  of  the  data,  as  well  as  to  permit  sharing  of 
interim  findings  of  the  studies. 
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The  Resource  Management  and  Analysis  Division  of  DPESO  will  serve  as  the  focal 
point  for  receipt  of  the  data  as  well  as  answering  questions  pertaining  to  the  data 
collection.  Questions  relating  to  hardware  data  should  be  directed  to  Mr.  Jack  Bemis;  on 
software  data,  to  Mr.  Lee  Schumacher.  Both  can  be  reached  on  703-756-2335,  or  Autovon 
289-2335.  Data  should  be  sent  to: 

Mr.  Jack  Bemis 
DPESO-XC 

c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 

Please  help  us  assure  widespread  participation  so  that  the  results  will  be  meaningful 
to  the  PRR  process.  Thank  you. 


296 


PRR  LESSONS  LEARNED 


•  Contractual  Coverage 

•  Obtaining  contractor  participation  in  and  support  of  the  PRR  requires 
contractual  coverage. 

•  This  requirement  should  be  spelled  out  in  a  Statement  of  Work  (SOW)  and 
placed  in  the  ED  contract.  (PRRs  will  be  conducted  during  this  phase  -  prior 
to  Milestone  III.) 

•  The  SOW  should  preclude  any  new  data  being  generated  -  use  existing  records, 
reports,  management  data,  etc.  as  evidence  to  validate  claims  of  readiness. 

•  Define  extent  of  coverage  expected  and  extent  of  government  participation, 
i.e.,  'N'  visits  of  'X'  personnel  for  'Y'  days. 

•  Define  who  will  conduct  PRRs  at  subcontractors  (prime  contractor  lead  with 
government  observing  or  government  lead  with  prime  contractor  observing. 
Spell  out  in  SOW). 

•  Include  guidance  regarding  which  subcontractors  will  be  visited  and  number  of 
visits. 

•  Planning 


•  An  effective  planning  effort  starts  prior  to  commencement  of  the  ED  phase. 

•  Begin  by  forming  a  planning  group. 

•  Determine  contractor  selection  criteria  (those  contractors  to  be  reviewed) 
based  upon  program  size,  complexity,  time  and  manpower  available,  criti¬ 
cality,  new  technology  involved,  and  contractor's  past  performance. 

•  Develop  PRR  team  organization  and  determine  types  and  quantities  of 
personnel  required  for  the  team. 

•  Identify,  select,  and  train  personnel  forming  the  PRR  team. 

Go  where  necessary  and  use  all  available  influence  to  get  high  quality 

members  on  the  team.  Don't  settle  for  second  best. 

Use  of  outside  consultants  adds  prestige  and  credibility  to  PRR. 

Make  maximum  use  of  personnel  from  outside  the  PMO. 

Assign  members  to  specific  panels;  designate  panel  leaders. 

Maintain  maximum  possible  continuity  of  team  membership. 

•  Determine  number  of  PRR  visits  to  each  contractor  and  subcontractor 
(function  of  program  complexity  or  problems  anticipated).  Incremental  PRRs 
have  proven  more  productive  than  one-shot  reviews. 

•  Establish  schedule  for  reviews;  coordinate  with  all  parties  concerned  as  soon  as 
schedule  is  constructed  in  order  to  facilitate  overall  scheduling  and  minimize 
number  of  visits.  Coordination  should  include: 

Contractor;  Program  Office;  Service  PESO,  DPESO,  and  cognizant 

Contract  Administration  Office  (CAO). 

•  Prepare  team  Work  Packages  (line  of  questioning  to  be  followed  and  specific 
questions  to  be  asked). 
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•  Assign  an  individual  to  be  responsible  for  administrative  matters  (travel 
reservations,  rental  cars,  lodging,  security  clearances,  and  funding). 

•  Incorporate  some  flexibility  to  accommodate  situation  changes. 

•  Orientation/Familiarization 


•  The  PRR  team  members  (and  the  contractors  visited)  should  be  instructed  that 
the  success  of  the  PRR  process  is  measured  by  their  ability  to  improve  the 
transition  from  development  to  production  and  correct  what  may  be  wrong  or 
omitted  in  a  timely  manner. 


•  PRR  Team 

Brief  PRR  team  members  on  background  and  history  of  program. 

Discuss  PRR  objectives,  modus  operandi,  and  schedule. 

Discuss  availability  of  relevant  data,  reports,  studies,  and  contractor's 
track  records. 

Discuss  responsibilities  of  each  principal  contractor  involved  in  the 
program. 

Review  Work  Packages  and  modify  as  necessary. 

Arrange  for  CAO  briefing,  covering  contractor's  past  history,  current 
experience  in  this  program,  other  concurrent  in-house  government  and 
commercial  contracts. 

Valuable  insight  into  the  readiness  of  the  system  under  review  can  be 
obtained  from  examining  the  contractor's  experience/ problems  in  produc¬ 
ing  a  similar  product(s). 

Evaluation  of  the  readiness  of  a  system  for  production  cannot  be  made  in 
isolation  from  other  contracts  and  activities;  the  contractor's  entire 
business  base  must  be  taken  into  consideration  when  weighing  facilities, 
manpower,  and  financial  matters. 

•  Contractors 

•  Hold  Pre-PRR  briefing  with  contractor. 

Explain  objectives  of  PRR. 

Discuss  PRR  team  composition. 

Determine  members  and  types  of  contractor  personnel  required  for 
interface  with  PRR  team. 

Discuss  types  of  data,  reports,  and  records  readily  available  from 
contractor. 

Review  proposed  questions,  agenda,  and  schedule. 

Determine  extent  of  prime  contractors'  involvement  in  review  of 
subcontractors. 

•  On-Site  Procedures 

•  Start  the  PRR  with  a  brief  "government  only"  meeting  which  includes: 
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CAO  briefing  covering  contractor's  capabilities  and  problem  areas,  and 
the  CAO's  assessment  of  the  program  status. 

Restatement  of  modus  operandi  and  administrative  matters. 

•  Receive  a  briefing  from  the  contractor  which  summarizes  program  status  in 
areas  to  be  covered  by  PRR  panels. 

•  Introduce  all  personnel,  government  and  contractor,  to  be  involved  in  the  PRR 
(identify  respective  roles  in  PRR). 

•  Do  not  let  contractor  under  review  dictate  the  course  of  the  PRR. 

•  Keep  formal  briefings  to  a  minimum. 

•  Maximize  time  available  for  panels  to  meet  with  contractor  personnel  and  for 
discussion  of  specific  areas  of  responsibility. 

•  Concentrate  on  collecting  sufficient  relevant  information  and  data  to  support 
firm  conclusions. 

•  The  PRR  team  leader  should  be  sure  he  makes  use  of  all  his  expertise; 
consultants,  laboratory  personnel,  cost  analysts,  etc. 

•  Time  can  be  saved  by  providing  the  contractor(s)  with  a  list  of  questions  and 
reviewing  the  answers  prior  to  visits. 

•  Make  maximum  use  of  existing  data;  do  not  require  new  data  to  be  generated, 
or  old  data  to  be  reconfigured. 

•  Do  not  mix  other  program  management  business  with  conduct  of  the  PRR. 

•  Set  aside  a  period  each  day  for  meeting  of  government  participants  to 
highlight  and  discuss  current  findings,  exchange  information  between  PRR 
panels  and  convey  to  appropriate  panels  any  investigative  leads  detected  by 
another  panel. 

•  Treat  skills,  expertise  and  corporate  memory  of  on-site  CAO  personnel  as 
valuable  resources. 

•  Use  questions  in  Work  Packages  as  guides  for  line  of  questioning  to  be 
followed,  not  as  a  script. 

•  Hold  exit  briefing  at  conclusion  of  each  PRR  visit.  Areas  of  concern  or  un¬ 
satisfactory  findings  should  be  presented  to  the  contractor,  who  should  be 
made  aware  that  corrective  action  on  his  part  is  expected.  Positive  findings 
should  also  be  noted  in  the  exit  briefing. 

•  Top  management  of  both  the  PMO  and  contractor  should  be  encouraged  to 
attend  the  exit  briefing. 

•  Areas  of  Investigation  During  PRRs.  This  listing  augments  the  PRR  criteria  or 

"Production  Readiness  Guidelines"  contained  in  enclosure  1  to  DoDI  5000.38. 

•  A.  Product  Design 

9.  The  software  for  the  integral  or  embedded  computer  subsystem  has 
reached  a  state  of  development  and  test  compatible  with  the  entire 
system  and  does  not  constitute  risk  to  either  meeting  the  program 
schedule  or  causing  extensive  design  changes  to  system  hardware, 
as  evidenced  by: 

a.  Low  and  stable  level  of  change  activity. 

b.  Acceptability  of  configuration  documentation. 

c.  Acceptability  of  programming  language. 

d.  Availability  of  reserve  memory  capacity. 

e.  Satisfaction  of  computation  time  requirements. 
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•  D.  Materials  and  Purchased  Parts 

8.  Where  appropriate  the  Defense  Priorities  System  is  being  exercised 
to  reduce  lead  times  and  insure  meeting  schedules. 

•  G.  Contract  Administration 

2.  There  exist  established  procedures  for  processing  engineering 
change  proposals  and  control  of  government  furnished  property. 


•  System  Cost 

PRRs  must  pay  more  attention  to  the  cost  factor  in  assessing  the 
readiness  of  the  system  for  production.  Cost  has  become  the  principal 
issue  at  DSARC  III  on  most  programs.  Many  PRR  teams  have  found  it 
profitable  to  include  a  cost  analyst. 

Cost  issues  which  PRRs  should  address  include: 

Has  the  contractor  identified  high-cost  items  and  processes?  Is  he 
working  to  develop  alternate,  lower  cost  items  and  processes?  Has 
this  effort  been  verified,  and  is  the  effort  effective? 

Who  are  the  critical  high-cost  suppliers  and  subcontractors?  What 
is  being  done  to  reduce  costs  in  these  areas? 

Does  the  contractor  have  an  active  Value  Engineering  (VE) 
organization?  Is  it  adequately  staffed  with  well-trained  personnel? 
Where  is  this  group  on  the  organizational  chart?  Is  there 
documented  evidence  of  their  success  on  prior  programs  in  the 
company? 

Does  the  contractor  have  a  Design-to-Unit-Production-Cost 
(DTUPC)  goal?  Where  does  he  stand  in  relation  to  the  goal?  What 
steps  are  being  taken  to  meet  or  better  the  goal?  Are  the  current 
DTUPC  projections  adequately  supported?  (Review  the  data  and 
rationale  used  to  arrive  at  these  projections). 

Does  the  contractor  demonstrate  a  sincere  interest  in  reducing 
costs,  or  is  he  just  paying  "lip  service"  to  the  overall  philosophy? 
Does  the  contractor  solicit  adequate  competition  for  purchased 
items? 

•  PRR  Execution  (In-House  and  Contractor) 

The  PRR  should  be  concerned  with  minimizing  the  costs  entailed  with 
the  PRR  itself  and  Government  PRR  costs. 

PRR  team  size  should  only  be  as  large  as  needed  to  do  the  job.  Too 
many  team  members  adds  costs  and  reduces  efficiency  of  other 
members. 

—  Hold  duration  of  reviews  to  the  minimum  necessary  to  obtain 
needed  data. 

—  Utilize  available  services  of  resident  CAO  personnel. 

—  Schedule  PRRs  by  geographical  areas  to  avoid  multiple  cross¬ 
country  trips. 
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Contractor  costs  in  support  of  the  PRR. 

—  Make  use  of  existing  data  and  resources.  Do  not  ask  for  special 
reports  or  data  to  be  generated. 

—  Encourage  the  contractor  to  make  available  only  those  personnel 
directly  concerned  with  areas  being  reviewed.  Discourage  large 
numbers  of  contractor  personnel  being  in  attendance  when  they  are 
not  directly  involved  in,  or  contributing  to,  the  review  process. 

—  Do  not  direct  the  contractor  to  take  any  action  which  could  be 
construed  as  a  constructive  change  and  a  cost  increase  to  the 
contract. 


•  PRR  Follow-Up 

•  Identify  problem  areas  and  areas  of  concern  in  PRR  Report. 

•  Document  findings  in  sufficient  depth. 

•  Pursue  action  items  generated  during  PRR  to  ensure  resolution  prior  to  the 
DSARC. 

•  PRR  report  should  identify  all  areas  of  risk  and  areas  which  are  unsatisfactory. 
Realistic  assessments  of  risk  areas  should  be  made,  identifying  potential 
program  impacts. 

•  Corrective  actions  taken  by  contractor  should  be  monitored  to  determine 
effects  of  these  actions. 

•  CAO  personnel  may  be  used  to  monitor  contractor's  corrective  actions  and 
report  on  same  to  PMO. 

•  Continuous  follow-up  activity  is  necessary  to  keep  pressure  on  those 
responsible  for  corrective  action. 
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LIST  OF  ATTENDEES 
PRR  CONFERENCE 
Wright  Patterson,  AFB,  OH 


Gerald  L.  Adams 

PM-DIVAD  ARRADCOM 

147  Combs  Hollow  Road 

Randolph,  NJ  07869 

Phone:  (AV)880-4170,  (CM)201-328-4170 

Don  G.  Alducin 

JCMPO  JCM-25 

Cruise  Missile 

Washington,  DC  20360 

Phone:  (AV)222-5035,  (CM)202-692-5035 

Christ  W.  Anagnost 
ARRADCOM 

ATTN:  DRDAR-PMA,  Bldg  151 
Dover,  NO  07801 

Phone:  (AV)880-6710,  (CM)201-328-6710 

Robert  J.  Baker 
Program  Manager's  Office 
XM-1  Tank  System 
Warren,  MI  48090 

Phone:  (AV)273-2661,  (CM)313-573-2661 

Truxtun  R.  Baldwin 

OSD  R&E  (AP)  MSA 

Pentagon,  Room  2A330 

Washington,  DC  21033 

Phone:  (AV)225-4258,  (CM)202-695-4258 

Michael  L.  Barbuschak 

COMNAVAIRSYSCOM 

ATTN:  PMA/PMS-266  (0P-1,  Room  710) 

Washington,  DC  20361 

Phone:  (AV)222-1367,  (CM)202-692-1367 

Oohn  C.  Bemis 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-233 5,  (CM)703-756-2335 

Kenneth  C.  Bezaury 

DRXIB,  Rock  Island  Arsenal 

ATTN:  IBEA-MT 

Rock  Island,  IL  61299 

Phone:  (AV)793-5235,  (CM)309-794-5235 


Robert  L.  Bidwell 

DoD  Product  Engineering  Services  Office 
c/o  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-2331,  (CM)703-756-2331 

Douglass  K.  Blood 
Program  Managers  Office 
XM-1  Tank  System 
Warren,  MI  48090 

Phone:  (AV)273-2661,  (CM)313-573-2661 
Alfred  E.  Bohnert 

Electronic  Warfare  Laboratory  (ERADCOM) 
ATTN:  DELEW-PE,  Bldg  2705 
Ft.  Monmouth,  NJ  07703 
Phone:  (AV)955-4135,  (CM)201-544-4135 

Robert  E.  Booth 

Naval  Air  Systems  Command 

ATTN:  AIR-5141B) 

Washington,  DC  20364 

Phone:  (AV)288-3350,  (CM)301-433-3350 

Morris  Bornstein 

ARRADCOM-LCWSL  DRDAR-XM 

ATTN:  Mortar  Systems  Project  Office,  Bldg  94 

Dover,  NJ  07801 

Phone:  (AV)880-2075,  (CM)201-328-2075 

James  R.  Brennan 

U.S.  Army  AVN  R&D  Cmd 

P.O.  Box  262 

Hazelwood,  MO  63042 

Phone:  (AV)698-1298,  (CM)314-263-1268 

Bernard  J.  Bretz 

U.S.  Army  MERADCOM 

ATTN:  DRDME-DE 

Fort  Belvoir,  VA  22060 

Phone:  (AV)354-5371,  (CM)703-664-5371 

William  R.  Briggs 

Space  Division/PMDM 

P.O.  Box  92960  Worldway  Postal  Center 

Los  Angeles,  CA  90009 

Phone:  (AV)833-0833,  (CM)213-643-0833 
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Paul  C.  Buck 

HQ,  Naval  Material  Command 
Code  08D3 

Washington,  DC  20360 

Phone:  (AV)222 -5884,  (CM)202-692-5884 

Thomas  L.  Campbell 
Directorate  of  Manufacturing/QA 
ASD/PMDP 

Wright  Patterson  AFB,  OH  45433 
Phone:  (AV)785-5541,  (CM)513-255-5541 

James  W.  Carstens 

US  Army  Industrial  Base  Engineering  Activity 
ATTN:  DRXIB-NP 
Rock  Island,  IL  61299 
Phone:  (AV)793-5113,  (CM)309-794-5113 

Robert  A.  Clark 

AVRADCOM,  Joint  Aeronautical  Materials  Activity 
ATTN:  AFSC/PMDM 
Wright  Patterson  AFB,  OH  45433 
Phone:  (AV)785-3307,  (CM)513-255-3307 

James  R.  Corwin 

Troop  Support  Aviation  Material  Readiness  Comman 

4300  Goodfellow  Blvd 

St.  Louis,  MO  63166 

Phone:  (AV)693-2220,  (CM)314-263-2220 

Frank  N.  Delillo 
HQ  BMO/MNCP 
Norton  AFB 

San  Bernardino,  CA  92409 

Phone:  (AV)876-6056,  (CM)714-382-6056 

Richard  R.  Delmar 

DARCOM/DRCMT  APESO 

5001  Eisenhower  Ave.,  Room  9N08 

Alexandria,  VA  22015 

Phone:  (AV)284-8299,  (CM)202-274-8299 

Donald  H.  DeLude 

Naval  Air  Systems  Command 

ATTN:  AIR-514 

Washington,  DC  20361 

Phone:  (AV)222-2850,  (CM)202-692-2850 

Roger  R.  DeVall 

U.S.  Navy  NAVAIRSYSCOM 

ATTN:  PMA266 

Washington,  DC  20360 

Phone:  (AV)222-1369,  (CM)202-692-1369 


Ronald  E.  Dionne 
ALMC,  Bldg  12500 
ATTN:  SACM 
Ft.  Lee,  VA  23831 

Phone:  (AV)687-3250,  (CM)804-734-3250 

Calvin  R.  Ditrick 

HQ  AFSC/PMDE 

Andrews  AFB 

Andrews  AFB,  MD  20334 

Phone:  (AV)858-7291,  (CM)301-981-7291 

Andrew  M.  Dobo 
ASD/PMD  (PESO) 

Wright  Patterson  AFB 
Dayton,  OH  45433 

Phone:  (AV)785-2761,  (CM)513-255-2761 

James  P.  Dwyer 

SD/PMDM  -  Space  Division 

P.O.  Box  92960 

Los  Angeles,  CA  90009 

Phone:  (AV)833-0833,  (CM)213-643-0833 

Joseph  N.  Dynan 
NAVPRO  -  GE  Company 
1000  Western  Ave 
Lynn,  MA  01910 

Phone:  (AV)478-4202,  (CM)617-594-3141 

Malcolm  C.  Edelblute 
HQ,  ASD/YZD 
Wright  Patterson  AFB 
Dayton,  OH  43433 

Phone:  (AV)785-6812,  (CM)513-255-6812 
Myron  J.  Files,  Jr. 

USAF  Electronic  Systems  Div  -  Mfg  Directorate 
ATTN:  ESD-IOM 
Hanscom  AFB,  MA  01731 
Phone:  (AV)478-3540,  (CM)617-861-3540 

George  J.  Fitzpatrick 

Commander,  CORADCOM 

ATTN:  DRDCO-PE-EC-1 

Ft.  Monmouth,  NJ  07703 

Phone:  (AV)992-32 77,  (CM)201-532-3277 

Lawrence  B.  Gallion 
Information  Spectrum  Inc. 

1745  Jefferson  Davis  Highway 

Arlington,  VA  22202 

Phone:  (AV)  N/A,  (CM)703-521-1050 
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Raymond  D.  Gilbert 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-2325,  (CM)703-756-2325 

Donald  F.  Goodale 
ASD/PMDPM 
Wright  Patterson  AFB 
Dayton,  OH  45433 

Phone:  (AV)785-4964,  (CM)513-255-4964 
Jack  F.  Harris 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-2325,  (CM)703-756-2325 

Mary  S.  Hart 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-2335,  (CM)703-756-2335 

Roy  3.  Hendricks 
ASD/TAD 

Wright  Patterson  AFB 
Dayton,  OH  45433 

Phone:  (AV)785-2436,  (CM)513-255-2436 

William  L.  Holland 

MERADCOM 

ATTN:  DRCPM-FM 

Ft.  Belvoir,  VA  22060 

Phone:  (AV)354-5384,  (CM)703-664-5384 

Edward  S.  Houston 

Ballistic  Missile  Office 

ATTN:  BMO/MNCPB 

Norton  AFB,  CA  92409 

Phone:  (AV)876-6611,  (CM)714-382-66U 

William  W.  Jeffers 

DoD  Product  Engineering  Services  Office 
c/o  DLA,  Cameron  Station 
Alexandria,  VA  22314 
Phone:  (AV)289-2325,  (CM)703-756-2325 

Lawrence  E.  Kennedy 

AVRADCOM,  Dir  for  Proc  &  Pdn,  Ind  Mgt/Pdn  Div 
4300  Goodfellow  Blvd. 

St.  Louis,  MO  63120 

Phone:  (AV)693-1268,  (CM)314-263-1268 


Joseph  A.  Kiel 

Naval  Air  Systems  Command  (AIR-514P) 
Department  of  the  Navy 
Washington,  DC  20361 
Phone:  (AV)222-2850,  (CM)202-692-2850 

Roger  N.  Koren 

Commander  USAERADCOM,  NV&EOL  Bldg  305 
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MANUFACTURING  READINESS 
FOR  PRODUCTION 


ducibility.  Producibility  considerations  during  preliminary  plan¬ 
ning  include  economic  evaluation  of  alternative  configurations. 


Richard  A.  Sipe 

IUS  Manufacturing  Manager 

Boeing  Aerospace  Company 

Manufacturing  readiness  for  production  starts  early  in  the  design 
phase  of  a  program  with  the  integration  of  design  and  production 
producibility  analysis  activities.  Increased  emphasis  is  being 
placed  by  the  Government  on  production  management  early 
involvement  in  the  product  acquisition  cycle  The  customer  today 
is  demanding  special  attention  to  decrease  production  costs  prior 
to  as  well  as  during  production.  To  meet  this  objective  a  planned, 
explicit,  and  timely  assessment  of  the  production  management 
implications  and  production  risks  are  necessary.  This  assessment 
covers  the  program  from  the  beginning  of  the  acquisition  cycle 
through  the  decision  to  go  into  production. 

There  are  a  number  of  required  guidelines  for  both  the  customer 
and  the  contractor.  One  of  the  foremost  involving  manufacturing 
is  MIL-STD-I528,  “Production  Management,”  describing  the  Air 
Force  requirements  for  an  effective  production  management  sys¬ 
tem.  This  MIL-STD  was  written  to  be  applied  in  total,  or  tailored 
to,  the  specific  requirements  of  each  contract  based  on  program 
technical  complexity,  production  risk,  production  cost,  scope  of 
production  operations,  and  potential  for  program  impact  due  to 
production  operations.  Implementation  of  MIL-STD-1528  typi¬ 
cally  requires  documentation  (a  formal  production  plan)  and  the 
conduct  of  formal  production  readiness  reviews  prior  to  transi¬ 
tion  from  full  scale  development  to  production.  The  following 
discussion  is  drawn  from  my  Boeing  manufacturing  operations 
experience  with  several  programs,  but  particularly  from  IUS. 

The  success  of  IUS  or  any  product  is  established  during  the  design 
phase  of  a  program.  For  example,  during  the  IUS  concept  (valida¬ 
tion)  phase,  Boeing  designed  the  upper  stage  vehicle,  developed 
and  tested  certain  high  risk  hardware,  and  conducted  many  major 
tradeoff  producibility  analyses.  Manufacturing’s  key  role  during 
this  early  phase  involved  fabrication  and  assembly  of  develop¬ 
mental  test  hardware,  initiation  of  a  producibility  program  for 
design  analysis  and  supportive  trade  studies  on  alternate  design 
concepts.  It  also  involved  participation  in  subcontractor  source 
selection  surveys,  preparation  of  preliminary  make/buy  plans,  and 
preparation  of  a  preliminary  production  plan. 

Certainly  a  key  activity  to  the  development  of  a  good  production 
plan  is  the  determination  and  optimization  of  the  product’s  pro- 


These  alternatives  could  be  any,  all,  or  a  combination  of  mate¬ 
rials,  methods,  processes,  technology,  tooling,  test  equipment  and 
procedures,  sequence  of  processes,  factory  layout  and  flow,  lot 
size,  cyclic  demand,  packaging  and  handling,  inspection,  manual 
versus  computer  part  processing,  and  capability  and  capacity 
versus  outside  source  development.  These  considerations  also 
include  review  of  drawings,  specifications,  test  procedures,  and 
tool  and  production  planning.  The  review  is  essential  to  verify 
that  all  necessaiy  information  is  included  for  the  craftsmen  in  the 
factory  to  fabricate,  assemble,  install,  and  test  parts  and  assem¬ 
blies  within  program  requirements.  This  activity  requires  a  coordi¬ 
nated  effort  between  the  designer  and  manufacturing  engineer 
responsible  for  a  hardware  package.  Theirs  is  a  continuing  rela¬ 
tionship  from  concept  through  final  determination  of  a  produc¬ 
tion  configuration.  They  are  supported,  as  necessary,  by  Materiel, 
Manufacturing  Technology,  Industrial  Engineering,  Quality  Assur¬ 
ance,  and  Factory  organizations  to  assure  that  all  elements  affect¬ 
ing  a  cost  effective,  reliable  end  product  are  identified  and 
incorporated. 

While  producibility  analysis  is  a  significant  activity  in  reducing 
production  risk,  it  is  not  always  a  precise  science.  It  is  more  of  an 
art,  where  perseverance  and  just  plain  hardnosed  experience  are 
the  key  to  success.  To  explain  what  I  mean,  let  me  relate  the  PDU 
chassis  story  .  The  power  distribution  unit  (PDU),  one  of  the  IUS 
electrical  power  distribution  system  electronic  boxes,  was  design- 
specified  as  a  welded  chassis  assembly  constructed  of  many  6061 
aluminum  detail  parts.  Producibility  analysis  of  the  PDU  chassis 
resulted  in  the  following- 

•  Reduction  in  part  count  by  32  (44-1 2) 

•  Reduction  in  manufacturing  flow  time  by  48  mandays  (9048) 

•  Reduction  in  linear  inches  of  weld  by  375  inches  (many  multi¬ 
ple  passes) 

•  Weld  repair  drastically  reduced  through  use  of  2219  aluminum 
material 

•  Heat  treat  eliminated  (age  only) 

•  Weld  engineer  surveillance  virtually  eliminated 

•  Factory  labor  manhours  per  unit  were  significantly  reduced 

•  Product  quality  and  appearance  were  greatly  enhanced 

This  story  illustrates  the  point  that  craftsmanship  in  the  factory 
must  be  backed  up  with  craftsmanship  in  design,  producibility 
analysis,  and  production  planning  to  achieve  optimum  end 
product  results. 
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Not  all  of  the  hardware  in  any  system  is  made  by  the  prime  con¬ 
tractor.  Attention  must  also  be  paid  to  the  subcontractors  and 
their  producibility  analyses  and  documentation.  Engineering, 
Manufacturing,  and  Quality  Assurance  provide  technical  support 
to  the  subcontractors  in  their  producibility  evaluations. 

As  noted  earlier,  one  of  the  Government  initiatives  promulgated 
through  MIL-STD-1528  is  the  requirement  for  formal  production 
plans.  Documentation  portrays  “Methods  and  concepts  for 
employing  facilities,  tooling,  and  manpower  resources  of  the  con¬ 
tractor  and  subcontractors.  It  reflects  all  time-phased  production 
actions  required  to  produce,  test,  inspect,  and  deliver  acceptable 
contractual  end  items  on  schedule  at  minimum  cost.” 

The  production  plan  usually  is  comprised  of  the  following 
sections: 

I  Manufacturing  Organization 
II  Make  or  Buy 

III  Subcontracting 

IV  Resources  and  Manufacturing  Capability 

V  Production  Planning 

Detail  requirements  for  each  section  of  the  production  plan  can 
be  found  in  Government  Data  Item  Description  DI-P-3460.  The 
significance  of  the  production  plan  is  that  it  provides  a  detailed 
“road  map”  from  drawing  to  completed  hardware.  It  incorporates 
the  results  of  producibility  analysis  and  trade  studies.  It  is  sub¬ 
mitted  to,  and  approved  by,  the  Government  and  updated  during 
various  program  phases.  Properly  done,  a  production  plan  is  the 
Manufacturing  person’s  bible. 

Using  the  production  plan  as  a  baseline,  you  can  verify  that  your 
homework  has  been  done  and  your  planning  is  complete.  This 
allows  the  contractor  and  the  Government  to  conduct  indepth 
reviews.  As  an  example,  I  will  describe  how  Boeing  and  Space 
Division/NASA  conducted  reviews  of  IUS  readiness  for 
production. 

DOD  Instruction  5000.28  describes  the  objective  of  a  production 
readiness  review  (PRR)  as  follows: 

“To  verify  that  production  design,  planning,  and  associated  prep¬ 
arations  for  a  system  have  progressed  to  the  point  where  a  pro¬ 
duction  commitment  can  be  made  without  incurring  unaccept¬ 
able  risks  of  breaching  thresholds  of  schedule,  performance,  cost, 
or  other  established  criteria.” 

The  criteria  for  conduct  of  PRRs  are  provided  in  AFSCP84-2  and 
overall  production  management  evaluation  approach  is  docu¬ 
mented  in  AFSCP84-3.  These  instructions  deal  with  functions 
that  are  highly  specialized  and  subject  to  differing  interpretations, 
definitions,  and  understandings.  Many  program/functional  groups 
in  Government/contractor  organizations  are  impacted.  Misunder¬ 
standing  of  interpretations  and  requirements  can  be  avoided  by 
very  early  joint  participation  by  contractor  and  the  Government 
in  developing  a  production  readiness  review  plan. 

The  Boeing/Govemment  team  who  conducted  1 1  PRRs  at  sub¬ 
contractors  supplying  mission  critical  hardware  to  IUS  were,  for 
the  most  part,  the  same  team  that  conducted  the  Boeing  PRR. 
During  the  subcontract  PRRs,  Boeing  conducted  the  review  with 
Government  team  members  acting  as  advisors.  During  the  Boeing 


PRR,  the  roles  were  reversed  with  Government  team  members 
conducting  the  review  and  Boeing  team  members  providing  the 
information.  This  is  an  effective  way  of  developing  an  integrated 
industry/Govemment  team  early  in  the  program.  It  provides  a 
mutual  understanding  of  system  elements,  functions,  and  inter¬ 
faces 'and  leads  to  an  efficient  and  economical  system-level 
review. 

I  believe  I  can  offer  a  few  “Lessons  Learned”  which  may  help  in 
demonstrating  manufacturing  readiness  for  production. 

Do. 

•  Understand  what  PRRs  are  all  about  and  the  significance  the 
customer  places  on  them. 

•  Posture  yourself  early  enough  to  be  ready. 

•  Review  and  prepare 

•  Agenda 

•  PRR  criteria  and  ground  rules 

•  Program’s  themes/messages 

•  A  single  thread  story 

•  Understand  all  other  elements,  functions,  and  interfaces 

•  Reflect  a  single  program  team  approach. 

•  Use  the  production  plan  as  baseline  for  the  PRR. 

•  Be  responsive  to  the  customer  and  understand  his  require¬ 
ments  at  all  times. 

•  Prompt  a  response-oriented  session  where  the  customer  will 
discuss  his  concerns. 

•  Balance  showmanship  with  substance. 

Don  7: 

•  Surprise  the  customer  in  a  PRR. 

•  Forget  to  be  prepared  to  address  problems  encountered  during 
developmental  phases  and  do  show  how  they  were  solved 
before  production. 

•  Indicate  everything  is  peaches  and  cream;  risks  do  exist; 
therefore, 

•  Identify  risks 

•  Show  good  abatement  plans,  and 

•  Show  that  risks  are  controlled. 

•  Forget  to  follow  up  on  questions,  comments,  or  last  PRR 
issues. 

•  Be  caught  on  MIL-STDs  (1695,  I520A,  1567,  1528,  etc.). 

•  Develop  a  proposal  one  way  and  really  plan  to  do  it  differently. 

•  Try  to  con  the  customer;  they  are  as  smart  as  you  are  and  have 
probably  been  through  this  more  than  you. 

•  Hipshoot  answers— get  back  with  specific  details. 

•  Give  impromptu  demonstrations;  if  a  demonstration  is  worth 
giving,  it  is  worth  preparing  for. 

•  Don’t  try  to  answer  questions  outside  your  area  of  expertise. 

•  Find  out  who  is  best  qualified  and  refer  the  questioner  to 
them.  If  you  can’t  help,  then  be  helpful. 

In  summary,  three  recommendations  are  offered: 

•  The  Contractor/Govemment  must  mutually  define  the  manu¬ 
facturing  readiness  criteria  early  in  the  program. 

•  Utilize  resident  Government  agencies  (AFPRO,  DCAS,  NASA) 
in  the  planning  and  conduct  of  production  readiness  reviews  at 
both  the  prime  contractor  plant  and  subcontractor  plants. 

•  Contractor/Govemment  must  both  do  your  homework;  know 
the  production  plan,  and  the  design  and  specification  require¬ 
ments  before  reviews  are  conducted. 
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PROGRAM  MANAGERS  GUIDE 


FOR 

PRODUCTION  MANAGEMENT 

One  of  the  major  recommendations  of  the  PRR  conference  was  to 
develop  mechanisms  for  the  dissemination  of  lessons  learned  on  PRR's. 
One  potentially  effective  vehicle  to  meet  some  of  this  need  is  the  up¬ 
coming  Program  Managers  Guide  for  Production  Management.  This  document 
is  being  developed  at  the  request  of  the  Joint  Logistics  Commanders 
through  a  contract  issued  by  the  Defense  Systems  Management  College 
to  Analytics. 

Part  of  the  effort  under  this  contract  is  to  identify  the 
current  practices  of  the  DoD  in  production  management  and  the  lesson 
learned  in  applying  production  management  techniques  to  DoD  programs. 
The  planned  use  of  the  Guide  is  as  a  desk  reference  to  be  issued  to 
Program  Management  offices  to  aid  in  the  management  of  acquisition 
programs.  The  first  draft  of  the  contents  of  the  Guide  is  attached. 
Your  assistance  in  providing  support  to  the  development  of  this  guide 
is  requested.  If  you  have  any  comments,  suggestions  as  to  coverage 
or  documentation  which  would  contribute  to  a  more  effective  document, 
please  contact: 


Mr.  Tom  McCann 
Analytics 

4124  Linden  Avenue 
Dayton,  OH  45432 

(513)  253-0010 
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APPENDIX  A 


Suggested  topics  for  inclusion  in  the  Program  Manager's  Guide  for  Pro¬ 
duction  Management: 

Chapter 

1.  PRODUCTION  MANAGEMENT  OVERVIEW.  The  industrial  base  and  its 
relationship  to  a  program.  Overall  production  concept.  Demand  and 
capacity.  Long  and  short  term  operational  and  investment  decisions 
of  the  contractor.  How  to  determine  program  production  requirements 
and  evaluate  contractor  capabilities.  Criteria  for  determining 
efficiency  and  effectiveness  of  a  production  management  system. 

Current  issues:  productivity;  energy;  OSHA  requirements;  environment; 
standardization;  inflation,  profits. 

2.  PRODUCTION  PREPAREDNESS.  Industry  vs.  government  objectives. 
Corporate  planning  and  business  environment.  Labor  skill  levels  and 
their  availability.  Cost-volume-profit  relationships.  Contracting, 
subcontracting,  make-or-buy  strategies  and  controls.  Competition  at 
the  contractor  and  subcontractor  levels.  Second-sourcing  decisions. 

3.  PROGRAM  MANAGEMENT  RESPONSIBILITIES.  Responsibilities  of  the  Ser¬ 
vice  program  manager  relative  to  production  management.  Selection  of 
personnel.  Responsibilities  of  the  industry  program  manager  relative 

to  production  management.  Nature  of  problems  at  the  Government/industry 
interface  and  how  to  cope  with  them.  Managing  contracts  when  a  second- 
source  is  involved.  Government  financing  and  management  of  multiple 
sources  of  production  critical  components.  Managing  when  NATO  countries 
are  involved  in  co-production  programs. 

4.  PREPARATION  FOR  PRODUCTION.  Product  development.  Production  plan. 
Concurrency.  Technical  risks.  Work-breakdown  structure  and  charting 
techniques.  Manufacturing  technology.  Introduction  of  new  processes. 
Purchasing  plan.  Quality  assurance  plan.  Maintenance  plan.  Resource 
requirements:  labor,  capital,  and  facilities. 

5.  PRODUCTION  CAPACITY.  Planning  facilities  for  production:  plant 
location,  plant  design,  plant  layout.  Contractor  incentives  to  build/ 
modernize  facilities.  Termination  buyback  techniques.  Contractor  vs. 
Government-furnished  facilities  and  services.  Rewarding  contractors 
for  investments  in  cost-reducing  production  facilities.  Idle  facility 
and  idle  capacity  costs. 
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6.  PRODUCTION  MANAGEMENT  TOOLS.  Information  systems  to  support 
decision-making.  Computer-aided:  management,  design,  manufacturing, 
inspection,  and  testing.  Applications.  Limitations.  Benefits. 

Industrial  robots.  Adaptive  and  non-adaptive  production  control 
systems. 

7.  PRODUCTION  CONTROLS.  Production  planning,  scheduling,  and  control. 
Budgetary  and  cost  control .  Performance  measurement  and  control .  In¬ 
spection.  Quality  Assurance.  Contractor  material  review  board  actions. 
Test.  Production  reviews.  Inventory  control.  Logistic  support. 

8.  PRODUCTIVITY  IMPROVEMENT.  Productivity  methods,  standards  and 
measurement.  Motivation  and  training  of  personnel.  Learning  curves. 
Capital  investment:  tooling,  equipment,  facilities.  Industrial 
engineering.  Value  engineering. 

9.  PROGRAM  CONTROLS  AND  DOCUMENTATION  RELATING  TO  PRODUCTION.  Con¬ 
tracting  provisions  and  incentives.  Reporting  systems  and  documentation. 
Production  management  controls,  including  1 ine-of-balance  and  production 
reviews.  Contractor  indirect  costs.  Financial  management.  Configuration 
management. 

10.  TRANSFER  OF  PROGRAM  MANAGEMENT  RESPONS IBIL ITEIS .  Policies,  criteria 
and  procedures  for  transferring  program  management  responsibilities  from 
a  development  to  a  readiness  command.  Problems  and  solutions. 

11.  CASE  STUDIES  IN  PRODUCTION  MANAGEMENT.  Army,  Navy,  and  Air  Force 
case  studies,  including  lessons  learned  in  various  facets  of  production 
management. 
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